Teaching SE: code as design

Software engineering lecturers have some misconceptions to grapple with — students’ certainly, but also our own.

One is this: we have tried to carve out an unambiguous1 distinction between software design and software implementation. In a previous post, I discussed a 2nd-year unit that historically focused on the Unified Modelling Language (UML). This unit purported to teach software design, to the exclusion of the “implementation” of that design. The unit content consisted almost entirely of diagrammatic notations (parts of UML) that, we assumed when they were first introduced, would stand on their own as the language of software design. The unit mentioned almost nothing about code, because we considered code to be a low-level implementation detail already covered in other units.

“UML is how you will write software”, some of us thought (to varying extents). “What comes after is just a labourer’s job.”

We2 wanted to believe this, probably because it seemed to imply that SE was maturing, that it was starting to develop efficiencies that would make current practice look like dark-age superstition. The notion of having separate “design” and “implementation” phases has been taken from older engineering disciplines (to some extent), in which a physical object is first designed, then constructed. That strict progression seems obvious. Design is the intellectual effort — the problem solving — while construction is physical creation of what you’ve designed.

Our UML fixation also owes something to the fact that diagrams play such an important role in older engineering disciplines. How would you construct a plane or a building without any diagrams to work from? You could do it, by describing measurements using only words and equations, but it would be painfully inefficient and error-prone. Why would you not want a picture of the thing you’re about to create? For software engineering, UML seems to slot naturally into this role.

We also noticed that software development requires you to think at different levels of abstraction. Initially, the work is highly abstract, big-picture stuff: whiteboard sketches and high level specifications, both of which have often been done with UML (or, historically, other diagrammatic notations). Work then progresses to more concrete and finely detailed stuff: code. There seems to be an obvious distinction here: a design phase, complete with diagrams, and a construction phase.

But this was always a conceptual mistake. Both activities are design, and there isn’t a construction phase at all (at least, not one that has any significant cost). The big-picture diagrams and the finely-detailed code are both purely intellectual; they’re both part of a single problem-solving exercise — one big design phase. The problem solving doesn’t stop until the coding is finished, and it often goes back on itself, with some whiteboarding required in the midst of coding. What would you think of the engineering of aircraft, buildings, etc. if you knew that the engineers routinely conducted critical redesign work half-way through physical construction? It’s not so much that software engineering is radically different3, but that we’ve been misusing the terms.

Real-world SE uses the term “building” or “build process” to refer to a set of automatable steps — compiling, linking, packaging, running unit tests — that turn the code into a finished product4. SE education forgets this a bit, but it’s a much closer analogue of the construction of a physical object than is the concept of the software “implementation”.

Once you accept that coding is, in fact, fundamentally a design activity, you realise that you cannot really teach software design without coding. If you take the coding out of software design, you’re really just left with an empty shell.

Our UML unit suffered because students were unable to connect UML’s diagrammatic notations to the things software actually has to do. Buildings and planes have schematics that broadly look like the physical object; there is an intuitive spatial relationship. But software has no tangible existence. A software diagram has nothing to do with what the software looks like, because software doesn’t look like anything. Software diagrams are no more than an aid to problem-solving. And how could we expect students to undertake problem-solving — design — without access to the one notation — code — in which the solution must be written?

Ultimately, code is everything. UML diagrams are merely subsets of the information present in code. They are abstract representations that highlight a few key details at the expense of many others. This can be very useful, but only as a way of organising your understanding of the code. Almost perversely, UML shows the big picture but never the whole picture. Experienced software engineers will understand the kinds of information omitted from a UML diagram, and thus what work remains to be done. But without code, students will look upon the diagram itself as the thing they need to master, without seeing it as merely a view of something more complicated.

Last year, alongside introducing patterns, I gave students a non-trivial amount of coding to do. (In hindsight, I probably gave them too much, to the point that it became a significant drain on their time, but such things can be recalibrated.)

In the weekly practical sessions, students formed into groups of three (or so). Over the course of the semester5, they worked on developing three pieces of software: an image viewer, library catalogue and a blog editor. Each system would be developed in a different language: Java, C++ or Python6, with the students choosing which language to assign to which system. Code, of course, is not just one notation, but many. The use of multiple languages was intended to show that software patterns (and other design concepts) apply to different languages and in slightly different ways.

Each week, I gave the students a few extra requirements for each system, drawing from the topics covered in that week’s lecture. This was all non-assessable — just preparation for the tests, assignment and exam.

A further twist was that students rotated between the different projects within their group, so that one group member would work on the image viewer one week, the library catalogue the next week, then the blog editor, then return to the image viewer, and so on. This would expose each student to three different languages, and also to the challenges of working with other people’s code. In practice, students became frustrated at the inability of their colleagues to actually finish their assigned tasks. That suggests I have some more thinking of my own to do, but I hope that these frustrations themselves served as a learning experience.

The unit still includes UML as a way of organising broad concepts. In fact, the introduction of code makes it easier to talk about UML. Students and I can use it to focus on the design concepts we need to convey, without getting too bogged down in superficial syntax or arcane rules.

By giving students the opportunity to write code, we give them all the information and the complexity that software design actually entails. Armed with that insight, they are in a better position to understand what design diagrams are for, and they can use them more effectively.

  1. Well, sort of. We occasionally pay lip service to the concept of an overlap. []
  2. Almost certainly not everyone believed this. Those who didn’t probably have some entitlement to point and laugh. []
  3. Software engineering is different — all engineering disciplines are different from each other, in terms of the methods and tools used — but we need not manufacture more differences than actually exist. []
  4. Or, rather, a potentially working product. The software build process is exceptionally cheap, because it’s automatable and has no material costs, so it’s done early and often. []
  5. In practice, mostly in the first half of semester, due to mounting stress levels and triaging of study effort. []
  6. There are many possible choices, of course, but I sought languages that were widely used and represented a diversity of approaches, but which basically adhered to the traditional OO paradigm. Java is our baseline teaching language (giving it the edge over C#). C++ stands out because of its pervasiveness and unique (if somewhat horrible) challenges. Python is a good representative of the dynamically-typed languages. []

Teaching SE: from UML to Patterns

In teaching software engineering at Curtin Uni, we have long had a 2nd-year unit that dealt principally with the grandiosely-named Unified Modelling Language (UML), that diagrammatic language that promised to be software’s answer to the technical drawings of other engineering disciplines. I recall it as a student, when the various UML notations (each one allowing you to articulate a particular aspect of the software you’re designing) were described at a rate of approximately one per lecture (with one lecture per week). At the time, not knowing any different, this seemed a logical approach to useful design material.

As a mathematical system, UML draws you in. Pure abstract curiosity drives you to understand what its constructs are for and how they relate to one another. It has a kind of internal elegance, if you know where to look, where a few core concepts can be used to articulate systems of great complexity with great subtlety and flexibility. Its creators made mammoth efforts to anticipate all the situations they wanted to be able model with a diagrammatic (or semi-diagrammatic) language, and provided the ultimate multi-purpose tool for the job.

UML positioned itself throughout the “software life cycle” (another curious term) but particularly in the design phase. It was supposed to be the vocabulary in which you expressed the bulk of your design work. You crafted UML diagrams from a requirements specification, then, once the UML “model” was complete, it became a blueprint with which you could hammer out the code in trivial time.

There are probably quite a few ways to explain why (in many cases or even most cases) this doesn’t really work. UML has a role in SE, but not one quite as celebrated as its evangelists might have thought. It is a vocabulary, one used to communicate complex, abstract ideas quickly. But not completely.

UML serves a useful role as an informal language, not a specification language, and the difference is completeness. UML can be complete — that was the whole idea according to some people — but practically speaking you don’t usually want it to be so. At the day-to-day level of human communication, you don’t need to be complete — you merely say as much as you need to say before the other person “gets it”. We trade off completeness for efficiency. When you want someone to understand what you’re thinking, you estimate the differences between your understanding and their understanding, and you communicate only those differences, not the sum total of your thoughts. In UML, you would draw only those entities and relationships that are relevant to the immediate issue at hand. (We take a very different approach when writing code and specifications, in which all details must be carefully articulated.)

In the second half of 2012, 11 years after I took the UML unit as a student, I was asked to teach it. The unit had evolved slightly in the way that units do, but the approach was broadly the same. Over the years, it had occasionally attracted the nickname “Diagrams 252”. People knew that the emphasis of the unit was misplaced. Now that I was on the other side of the educational fence, I came to see the problem in our teaching approach — not just in what we were telling students about UML, but in what we were failing to tell them about software design more generally.

UML is a vocabulary, formal or informal, but so what? Software design isn’t about syntax. It’s about ideas. What should you actually do with all those notations? Where was the actual design? Where was the engineering?

In many an SE textbook, there is no shortage of advice on what you shouldn’t do in software engineering. There are rules about how not to use inheritance, what language constructs you should avoid at all costs, what kinds of coupling are unacceptable, etc. There is usually not a great deal of advice about what you should or even can do.

Exposing students only to a set of rules and notations, as we did previously, is really just fiddling around the edges of SE education. The knowledge that creates a competent software engineer is not UML or design checklists. Rather, a competent software engineer first and foremost needs a mental toolbox of design options — a large collection of adaptable solutions to small problems, stored in one’s brain. These are not memorised in the way that students often try to memorise bullet point lists. They are not catalogued in a neat little taxonomy as might appear in a traditional textbook. They are haphazard fragments of possible design approaches — the result of seeing how design could be done in various ways. The technical term from cognitive science, as I understand it, is “plan”. Plans are building blocks for the creative and investigative processes.

One can acquire such a mental toolbox through sheer brutal personal experience — that’s how they did it in the old days, of course. But education is supposed to short-circuit that, or else what’s the point?

In teaching the unit for a second time in 2013, I made some significant changes, one of which was to teach software design patterns1. It’s easy to make the argument for teaching patterns, because they have that aura of academic respectability that accompanies things Published by Reputable Authors. Patterns, of course, are broadly defined as reusable solutions to common problems. They are implemented independently by different engineers in different contexts, and later discovered for what they are. Familiar?

The logic here is more convoluted than it first appears. A pattern is not a plan — not exactly. Each pattern is a named phenomenon (the “Strategy Pattern”, “Observer Pattern”, “Composite Pattern”, etc.), analysed, documented with UML and catalogued. The widely-known seminal list of software patterns was laid down in a book by the “Gang of Four”. So, for educational purposes, patterns sound a lot like a giant list of bullet points that students would simply try to memorise.

But patterns are important for educational purposes because (unlike UML itself or the usual what-not-to-do list of SE design rules) they show you what you can do. Patterns are solutions known to work well, assuming you understand what each one is for. Moreover, it has been my hope that teaching patterns also reveals some of the more fundamental ideas about software design. That is, even if you can’t remember a single actual pattern after taking the unit, you might nonetheless remember some interesting things you can do with polymorphism. Hopefully some knowledge underlying those patterns will have rubbed off, and given you the beginnings of that mental toolbox.

  1. There had always been a single lecture on patterns anyway, but it didn’t cover much material. []

To be a male feminist

I haven’t ranted in a rather long time, so here goes.

A (male) friend of mine voiced an opinion recently that, surely, everyone ought to be a feminist. At least, I shall rephrase slightly in deference to those with the greatest experience on the subject, everyone ought to aspire to feminism. It is a perspective, after all, not a club. (By “feminism”, I mean the struggle for equality, of course, and not the absurd man-hating caricature, mislabelled as “feminism”, that functions as a piñata in certain male-dominated political circles.)

Except — and this is where it gets tricky — feminism is actually a collection of perspectives, some of which are mutually contradictory, and not all of which acknowledge the existence of the others.

And then there’s another, stranger term — “feminist ally”. Say we define feminism to be a movement seeking fairness and equality between genders1. A feminist, then, is simply someone who believes in and supports this cause. So what is a “feminist ally”, if not an actual feminist? In what sense can someone outside the movement, who then by definition doesn’t believe in and/or support the cause, be an ally of it?

They can’t, so the term presupposes a different, stricter definition, one that excludes people on grounds other than their beliefs and actions. The most obvious of such exclusions is gender-based — you can’t be a male feminist.

Opinions seem divided on this. On one hand, some (whatever their gender) subscribe to the reasoning above and say that men can, and in fact should, be feminists. On the other hand, some (who are, again, not necessarily of one gender) reserve the classification for those with a more direct, personal stake in the struggle — i.e. women — and that a man claiming to be a feminist is portraying himself as a part of the oppressed group. Men supporting feminism are thus to be described as “feminist allies”. This contradiction neatly removes any possibility of a “safe” answer with respect to whether I, as a man, am a feminist (other than just keeping my head down and pretending it’s someone else’s problem, which it isn’t — I’ll come back to that).

So, I must make a choice, to the best of my cognitive abilities. While women (maybe some more than others) are clearly invested in feminism in ways that men never can be, feminism is still a cause that one joins by choice, not an underprivileged demographic in itself. For those within the feminist movement who suffer oppression, violence and discrimination, this is unlikely to be due to their being a feminist, but rather simply being a woman2. We shouldn’t conflate the words “feminist” and “woman”, and we should be able to deal with the notion of a “male feminist” without imagining that the man is necessarily equating his personal situation to those of women. (If he is, then that’s poor form, but labelling himself a “feminist” is a bizarre way to do it.)

This is all very theoretical so far, so let’s turn to the more concrete. I’m a software engineering lecturer, and in spite of all efforts thus far, female participation in the discipline is absurdly low. The first-year classes I teach tend to have around 100 to 150 students, only 5 to 8 of whom are women. I am the first to admit that this is a shameful situation, given my (perhaps biased) view that software engineering increasingly underpins the infrastructure on which the whole of Western (and even non-Western) society depends.

Some might retort that, well, so what? Universities in Australia admit whoever they can. There are no particular university policies that act to filter out women. When it comes to a choice between (a) sexism or (b) more money, university management displays pragmatic enlightenment. Female participation in many university courses is actually very high. (I know this just from observing the disparity between my classes and other, non-computing-related classes during the exam period.) So, if women chose not to enrol in software engineering, that’s their choice.

But it’s an uncomfortable reality, because without knowing why it happens, we could be ignoring a festering problem.

It seems implausible to me that the gap could be the result of “different brain types”. We’re the same species, and it just doesn’t make sense for the same DNA (of homo sapiens), fighting for its survival (in evolutionary terms), to rob one gender almost entirely of the ability to conduct creative, abstract reasoning, which is what software engineering entails. Even allowing for gender roles in prehistoric societies, how could such deprivation possibly increase our overall odds of survival? (I realise that there is a wealth of research on the subject, but I haven’t ploughed into it myself.)

Of course, the existence of women authors, artists, scientists, etc., and the feminism movement itself, seems to roundly contradict that notion. If women can take on those jobs, they can be software engineers too. The women who do study software engineering perform no worse, as students, than their male counterparts, and some routinely rank among the best.

The issue, then, is almost certainly societal in origin, and that should be concerning. Even if we say that the gender ratio in software engineering is not a problem in itself, it still provides strong evidence of some underlying imbalance in society at large.

The best explanation I have (and I’m hampered by a complete lack of first-hand experience) is that girls in high school, and perhaps even primary school, are dissuaded from studying technical subjects like maths, science and computing due to a persistent, self-reinforcing myth that these subjects are simply “for boys”. The myth of different brain types may be a sort-of self-fulfilling prophecy.

So, getting back to the original point, am I a feminist? I certainly aspire to feminism. Perhaps the litmus test, depending on who you ask, is whether I am actually making a difference. Unfortunately, I can’t point to any particular evidence of this yet. We shall see.

  1. I think a majority of people would broadly agree with this definition. Some might wish to clarify or extend it, but I don’t need a particularly finely-tuned definition here to make my point. []
  2. I’m sure there are cases of discrimination on the basis of feminist beliefs and actions, separate from any gender consideration, but I would be surprised if this was comparable in scope to discrimination against women generally. []

Non-consensual wisdom

Previously, Shane Greenup brought to my attention two very interesting software projects, with somewhat similar goals: his own rbutr (currently in beta testing), and Dan Whaley’s Hypothes.is (currently being planned and prototyped).

Rbutr (pronounced “rebutter”) allows its user base to link together web pages that rebut one another. These links eventually form conversation chains and webs that may span any number of websites, without needing or seeking the consent of the website owners. I, as a blogger, would have no control (or, at least, no veto) over rbutr links connecting my blog posts to someone else’s refutation of them, but these links would be available for any reader (who uses rbutr) to see and follow.

Hypothes.is has the even more ambitious goal of providing an “annotation layer” for the Internet. Any arbitrary passage of text (as well as other media types, including images, video and audio) within any web page may be adorned with a critical remark, visible to anyone else using the software, again without the consent of the web site owner. It aims to be a fine-grained peer-review system for, well, everything.

The minds behind Hypothes.is are open about the fact that others have tried and failed (to varying extents) in the goal of creating a “web annotator”. However, they seem very determined to identify and learn from past mistakes. Perhaps the most important of these has been the lack of quality control in creating annotations. In a previous post I mentioned a similar project called Dispute Finder. I now gather that Dispute Finder’s database itself may have been overrun by misinformation. As one article explains:

Third, and most critical in my thinking, there will be [in Hypothes.is] an extensive reputation system to qualify and rank comments based on the expertise of the commenter. The lack of this was part of what doomed an earlier project called Dispute Finder. I thought for a while that it would evolve into the tool skeptics needed, but very quickly the data in that tool was awash in conspiracy theories and other nonsense, with no way provided to sort by quality.

Hypothes.is is bringing together a pool of experts to determine how to create a “reputation model” to prevent this sort of thing from happening again. After all, Wikipedia seems to manage commendably well to resist incursions from interest groups1. Even the Slashdot moderation system seems to successfully raise up interesting and insightful comments at the expense of mundane and simplistic ones. I feel that our collective intelligence, though sometimes disorganised, is often under-appreciated.

Projects like this might prove an attractive middle road between (a) the Internet as a anarchic incubator of (mis-)information, and (b) the Internet as an oppressively-sanitised, centrally-regulated newspaper. Join the dots, for instance, between Hypothes.is and the current debate over media regulation in Australia. Libertarian-minded newspapers and bloggers take furious offence to any suggestion that their activities should be overseen by The Government.

It would be hard to mount quite the same argument, with quite the same emotive imagery, against Hypothes.is or rbutr. While non-consensual, there is certainly no coercion involved — no fines, no censorship, no forced apologies, etc. There is nothing here that need be sanctioned by those in power. The system operates on a purely informative level. Affected websites are not required to do anything, and nobody is required to use the system in the first place. Such systems can only succeed if people choose to use them. That (presumably) will only happen as long as they meet a socially/psychologically acceptable level of reasonableness and transparency.

But neither is Hypothes.is or rbutr a “toothless tiger”. It would surely be a blow to authors’ and editors’ egos and credibility to have third-party corrections publicly scribbled over their otherwise majestic prose. They would have to contend with new, publicly-known metrics that assess aspects of their intellectual integrity, not just “hits” and the like that demonstrate their popularity. They would no longer enjoy the same flexibility with the truth, considering that their errors may be almost immediately visible. Any third-party annotations could easily become the most attention-grabbing parts of an article, destroying at a glance whatever the original (accidental or deliberate) misinformation may have been.

As a result, there would surely some backlash from tabloid newspapers and bloggers upon discovering that they no longer have absolute control over what their readers read when visiting their sites. They might even consider it a threat to their business model. Operators like Andrew Bolt certainly seem to make a career out of saying things that need to be corrected (while at the same time exhibiting extraordinary defensiveness).

If it works, Hypothes.is could initially make a lot of people very, very angry. There could be lawsuits — particularly of the defamation variety, I imagine — and that could be a problem for a non-profit organisation. But, if it gets that far, the idea of a peer-reviewed Internet has already won.

 

  1. That isn’t to say I’d rely on the quality of Wikipedia, necessarily. However, for a publicly-editable resource, it is curiously bereft of the kind of backhanded misinformation and puerile simplicity you find in many — even professional — online news articles or blog posts, and the outright lunacy you find in the comments section underneath (present company excepted, of course). []

Curing viral misinformation

A great deal of mischief is caused, regularly, by viral misinformation. Factoids that support one side of any controversial issue are rapidly copied and pasted many times over (the “echo chamber”). By the time anyone manages to marshal the truth into a coherent response, it’s too late — the lie has convinced enough people for it to become self-reinforcing. Everyone can probably name some examples of this, particularly in day-to-day national politics.

I can’t help but quote Churchill:

A lie gets halfway around the world before the truth has a chance to get its pants on.

(Given the Internet, this actually seems rather conservative.)

For me, the frenzied reaction in 2009 to the hacked CRU emails springs to mind. All manner of nefarious interpretations were placed on isolated snippets of private correspondence of climate scientists, before anyone in a position to understand the emails’ context (or at least the lack thereof) could conduct an honest evaluation. And in cases like this, the lies are often more complete than the truth, and certainly more interesting.

I don’t have an exact model of how this process unfolds. However, I suspect that, if we sat down and analysed a sample of propagated misinformation, we’d find that important parts of the original wording have largely been preserved, with very little paraphrasing. Misinformation only manages to propagate so fast because higher cognitive levels1 are (probably) never reached in the initial hours of propagation. This means that the propagation of misinformation is largely a mechanical process (not a creative one), which places it within the reach of automated or semi-automated analysis.

To come to the point, we can and should devise a tool to automatically detect this misinformation, and build it into the web browser — a browser extension. It should highlight and annotate misinformation in any web page the user views, based on a regularly-updated database. There are a few sites already dedicated to correcting misinformation (Snopes, Skeptical Science, etc.), and they are certainly invaluable, but a greater prize is to have misinformation annotated without any immediate human effort at all.

I’ve been toying with this idea for over a year, considering how to engineer communication between the browser extension and the database, how to provide flexibility in searching for different types of misinformation, while avoiding software security vulnerabilities, etc. (I should probably have written a prototype by now, but paid work took priority.)

It turns out — unsurprisingly — that others have considered some of these issues as well. The existing research tool Dispute Finder is very similar to what I’d envisaged. (It was well reported back in 2009, but clearly escaped my attention at the time). However, that project has apparently ended, and its principal investigator Rob Ennals has moved on. The Firefox browser extension has been removed, so I haven’t seen it in action, and presumably the database is no longer available either. The project did get as far as conducting user evaluations of the software. Perhaps Dispute Finder was only intended to have a fixed lifetime, or perhaps the authors decided that the project was not sufficiently successful.

Skeptical Science has its own Firefox browser extension, but this is climate-change-specific, and so is most likely to be used by those who consciously and actively accept the reality of climate change. That’s not to say it isn’t useful, but its effects on public discourse are probably indirect.

A generic “lie detector” tool might have a disproportionately greater impact on public discourse compared to a domain-specific tool. The generic tool would cover a much greater array of misinformation, and as a result would probably also gain wider acceptance. For instance, at least some of those who don’t particularly care about or believe climate science may nonetheless choose to use the generic tool for its treatment of other issues. (Hard core denialists of any stripe may complain about the “anomalous” treatment of their pet topics. Such complaints might be a blessing in disguise, actually boosting awareness.)

In fact, there are really two pieces of software here: the browser extension itself and the database. Given an appropriate means of communication, they could be developed quite independently.

The source code for Dispute Finder (previously “Think Link”) seems to be available here. I still intend to write my own independently, because I have different views on the technical architecture, which I may elucidate in future. The research findings of the Dispute Finder / “Confrontational Computing” project are certainly worth pondering, though. It would be a waste to ignore the experience gained, and it seems too good an idea to give up on.

  1. Bloom’s taxonomy breaks cognition into distinct levels: knowledge, comprehension, application, analysis, synthesis and evaluation. The “knowledge” level is pure rote learning, while “evaluation” represents critical thinking. []

One brand to fool them all

As you might have realised, I work at Curtin University, formerly Curtin University of Technology (CUT), formerly – though conceivably somewhat apocryphally – Curtin University of New Technology (CU*T), formerly the Western Australian Institute of Technology (WAIT), formerly Perth Technical College, formerly Perth Technical School, formerly – and definitely more apocryphally – the New Holland Colonial Blacksmith and Breakfast Bar, formerly the East Gondwana School of Blunt Instruments.

We’re not good at names.

One of the most frustrating aspects of this particular institutional blight is how it plays out in the University’s ICT services. There are those who must spend long dark hours of their lives dreaming up grandiose names with which to inspire the huddled masses to come forth and be dazzled by yet another online service. The problem is that we have hundreds of such services, and it’s an act of cognitive warfare to suggest that we should memorise that many bizarre acronyms and cute but hideously overly-generic terms and the circumstances in which they must be applied. Without wishing to blame anyone in particular, it’s all getting a bit ridiculous.

You can see how and why it happens. The University’s ICT infrastructure has grown organically, bits and pieces being added over time with no real coordination. This is probably inevitable in a large, diverse organisation. The plethora of different ICT services resemble a market, with each different product competing for mind share. However, it’s not a market, and in theory we’re supposed to use all of the relevant services. So, when a new service is added or updated, it suddenly becomes Very Important that everyone bow down before the mighty ingenuity involved, and recognise the sudden urgency with which the new technology must be adopted. The next service to be added or updated after that requires the same thing, and so on. To make it happen, each of these new services can’t just be named – they must be branded. ICT services are not just provided at Curtin – they are, in the marketing sense, sold.

Several years ago, University management commissioned the “OASIS” website, with the aim of integrating all the disparate online services (and introducing our beloved Official Communication Channel). OASIS originally stood stands for “Online Access to Student Information Service” (a backronym, one presumes). Now, however, it doesn’t seem to stand for anything. It’s just a meaningless name, and thus is itself a perfect example of the problem at work.

OASIS was originally marketted as the “One Site to Rule Them All”, which it sort of is, but only at a very superficial level. There’s a lot of delegation involved, and the “ruling them all” bit only goes as far as logging in. Once you’re logged in, you still need to navigate a maze of services that are still essentially separate niche applications. The fact that these are not fully integrated, functionally and stylistically, is not the first problem. The first problem is what they’re all called.

Names of these services include “eVALUate”, “StudentOne”, “eStudent”, and “eAcademic”, among others. My point is perhaps more easily grasped by an outsider, for whom these names must seem rather useless as descriptions of what the systems actually do. Indeed they are – eVALUuate has nothing to do with grading or student results, StudentOne is inaccessible to students, eStudent provides nothing that students will find useful on a regular basis, and eAcademic is actually used to access student information. (The true functions of these systems are, of course, better understood by Curtin staff and students than by outsiders, but only by being forced to use them.)

Now, there are many ways in which these services might be better integrated, but a not-insignificant amount of confusion and cognitive waste could be alleviated simply by coming up with names that actually make sense. By this, I mean intuitively obvious, not requiring large-scale internal marketing programs. The ICT branding we have at the moment is a complete waste of resources at every level. In my ever-humble opinion, all these services should have purely functional names. They should not stand out. They should not be cute, or cool, or inspiring, or grandiose. They should be simple, accurate, no-nonsense descriptions of the services provided. For instance:

  • OASIS should be called “Curtin online services”
  • eVALUate should be called “Course/teaching feedback”
  • StudentOne should be called “Student database”
  • eStudent should be called “View/update your enrolment details”
  • eAcademic should be called “View student details”

At least, that gives you some idea.

I don’t know how the mind of a marketing person might react to this. I’d hope that a good marketing person might recognise the merits of functional naming as a means of encouraging the use of ICT services.

Existence continuation

As you have doubtless deduced from my total failure to keep you entertained over the last month and a half, I have in fact been a little busy. Possible illusions to the contrary notwithstanding, my existence is not synonymous with that of my blog. (At least, not yet it isn’t. This may change later in life when my consciousness is uploaded. I’ll keep you posted.)

First, it may be worth noting that, after more than six years, my PhD thesis is about to be officially approved. I shall thus enter the PhD afterlife, my soul having been judged and marked, and corrections thereto proclaimed. I shall wander the Earth instilling great wisdom in anyone mildly curious about the nature and mechanics of comparisons between different software inspection strategies, for such has been the tiny sliver of human experience to which I have contributed.

Second, I have simultaneously rewritten and lectured a unit on C programming in UNIX, a feat probably not entirely without precedent, although this has been my first real lecturing experience. It is said that writing your own lecture notes gives you much better preparation for lecturing than reusing someone else’s, and this was probably true. In hindsight I would recommend having all this done before the semester begins, though in any event I didn’t have much choice. Over the course of 14 weeks, I developed 10 lectures, 9 tutorial worksheets and associated tutors’ notes, 3 tests (including one exam) plus marking guides, 3 mock tests plus answer guides, and 1 assignment plus marking guide. That’s 26-42 distinct documents (depending on how you count them), whose combined content would rival a PhD thesis (and I know). This was in addition to 24 hours/week of actual face-to-face teaching.

Third, (as any teacher or lecturer will be all too aware) after the teaching comes the marking. Exam marking is easy – there is no feedback to give, and a fairly constrained context for creative idiocy. You get to see what students do well, and what they do poorly, and what questions could be improved for next time. Prac report marking is also relatively easy, though collating them all at the end seems to lead to the conclusion among some students that they just weren’t going to be marked at all.

Assignment marking is hard. In my case, this is partly because programming assignments do not really constrain creative idiocy. There are many ways to write a program correctly, but infinitely more ways to do it badly. It’s not as mind numbing as (I imagine) essay marking must be. The very worst essays (such as the incoherent, smudged dribblings written in high school English Lit exams by none other than yours truly), must be positively soul destroying to read and mark. You can mark a programming assignment partly by running it, but nevertheless, like an essay, it must also be read.

Reading said programming assignments was made at the same time more entertaining and more frustrating by a discovery I made towards the end. Imagine, if you will, twelve student essays (essays being an analogy here for programs) that are identical except for the substitution of synonyms, punctuation, fonts and paragraph breaks. Then imagine the students involved swearing blind that they merely discussed the topic and certainly never copied anything, and that of course they were the same because the question could only be answered one way.  Then imagine – and this is what really annoys me and my good colleague who was roped into conducting the investigation – that a large proportion of these students are, for want of a better term, good students and will easily pass the unit. (At least, they will if my suspicions about the severity of their punishments turns out to be correct.)

Now that’s all behind me, I come to my last (and continuing) major task for the year: trying to find a job. Then, absurdly, I might have some free time.

Open source science

Slashdot notes an article from the Guardian: “If you’re going to do good science, release the computer code too“. The author is, Darrel Ince, is a Professor of Computing at The Open University. You might recognise something of the mayhem that is the climate change debate in the title.

Both the public release of scientific software and the defect content thereof are worthwhile topics for discussion. Unfortunately, Ince seems to go for over-the-top rhetoric without having a great deal of evidence to support his position.

For instance, Ince cites an article by Professor Les Hatton (who I also cite on account of his recent study on software inspection checklists). Hatton’s article here was on defects in scientific software. The unwary reader might get the impression that Hatton was specifically targetting recent climate modelling software, since that’s the theme of Ince’s article. However, Hatton discusses studies conducted from 1990-1994, in different scientific disciplines. The results might still be applicable, but it’s odd that the Ince would choose to cite such an old article as his only source. There are much newer and more relevant papers; for instance:

S. M. Easterbrook and T. C. Johns (2009), Engineering the Software for Understanding Climate Change, Computing in Science and Engineering.

I stumbled across this article within ten minutes of searching. While Hatton takes a broad sample of software from across disciplines, Easterbrook and Johns  delve into the processes employed specifically in the development of climate modelling software. Hatton reports defect densities of around 8 or 12 per KLOC (thousand lines of code), while Easterbrook and Johns suggest 0.03 defects per KLOC for the current version of the climate modelling software under analysis. Quite a difference – two orders of magnitude, for those counting.

Based on Hatton’s findings of the defectiveness of scientific software, Ince says:

This is hugely worrying when you realise that just one error — just one — will usually invalidate a computer program.

This is a profoundly strange thing for a Professor of Computing to say. It’s certainly true that one single error can invalidate a computer program, but whether it usually does this is not so obvious. There is no theory to support this proclamation, nor any empirical study (at least, none cited). Non-scientific programs are littered with bugs, and yet they are not useless. Easterbrook and Johns report that many defects, before being fixed, had been “treated as acceptable model imperfections in previous releases”, clearly not the sort of defects that would invalidate the model. After all, models never correspond perfectly to empirical observations anyway, especially in such complex systems as climate.

Ince claims, as a running theme, that:

Many climate scientists have refused to publish their computer programs.

His only example of this is Mann, who by Ince’s own admission did eventually release his code. The climate modelling software examined by Easterbrook and Johns is available under licence to other researchers, and RealClimate lists several more publicly-available climate modelling programs. I am left wondering what Ince is actually complaining about.

Finally, Ince seems to have a rather brutal view of what constitutes acceptable scientific behaviour:

So, if you are publishing research articles that use computer programs, if you want to claim that you are engaging in science, the programs are in your possession and you will not release them then I would not regard you as a scientist; I would also regard any papers based on the software as null and void.

This is quite a militant position, and does not sound like a scientist speaking. If Ince himself is to be believed (in that published climate research is often based on un-released code), then the reviewers of those papers who recommended publication clearly didn’t think as Ince does – that the code must be released.

Ince may be convinced that scientific software must be publicly-auditable. However, scientific validity ultimately derives from methodological rigour and the reproducibility of results, not from the availability of source code. The latter may be a good idea, but it is not necessary in order to ensure confidence in the science. Other independent researchers should be able to confirm or contradict your results without requiring your source code, because you should have explained all the important details in published papers. (In the event that your results are not reproducible due to a software defect, releasing the source code may help to pinpoint the problem, but that’s after the problem has been noticed.)

There was a time before computing power was widely available, when model calculations were evaluated manually. How on Earth did science cope back then, when there was no software to release?

Please reboot the aircraft

I was hearing vague snippets of the disaster that was the Virgin Blue computer system, but my JetStar flight had its own problems. Everyone was seated (that is, except for the restless and very, very sensitive toddler standing on the opposite window seat, who burst into tears whenever mum dared suggest he sit down and put his seat belt on), but there seemed to be a delay.

It was getting quite stuffy, actually. A couple of people took to fanning themselves with the A320-232 safety instruction cards. It emerged that there were “maintenance issues”, which sounded a little dubious. Shortly thereafter, the captain (or someone) informed us that the problem was indeed related to the air-con. He could fix it in 2 seconds, but he would need to switch the plane off.

Had they, on the spur of the moment, installed a new air-conditioning software update? At least this was happening before takeoff, I thought to myself. (For instance, they didn’t say this: “Sorry, ladies and gentlemen – we will shortly begin a rapid descent towards to ocean while we install this critical software patch and restart the aircraft. Not sure how long we’ll be – let’s just hope it works this time.”)

So, for about a minute, the cabin lights were replaced by blue-tinted torch light, the engines died down and there was eery quiet (that is, except for the gentle snorting of the person next to me and the squeals from across the isle). It was also a reprieve from the terrible, cheesy music that had been playing over the speakers to pass the time; cheesy to an extent that can surely only be achieved with premeditated malice.

Then, with our air-con software apparently working as advertised, all hands reached for the vents above our seats and we were off.

Admit me to the conspiracy

Deltoid takes a look at a piece of code taken from the Climate Research Unit (CRU) that apparently has the denialists salivating. Buried therein is the following comment: “Apply a VERY ARTIFICAL [sic] correction for decline!!” Are you convinced yet of the global leftist socialist global warming alarmist conspiracy?! I certainly am.

I’d also like to apply for membership. You see, trawling through my own code for handling experimental data (from September 2008), I’ve re-discovered my own comment: “Artificially extends a data set by a given amount”. Indeed, I appear to have written two entire functions to concoct artificial data*, clearly in nefarious support of the communist agenda. I therefore submit myself as a candidate for the conspiracy. The PhD is only a ruse, after all. Being a member of the Conspiracy is the only qualification that really counts in academia.

* I’m not making this up – I really do have such functions. However, lest you become concerned about the quality of my research, this artificial data was merely used to test the behaviour of the rest of my code. It was certainly not used to generate actual results. I can sympathise with the researcher(s) who leave such untidy snippets of  code lying around, and I’m a software engineer who should know better!