Seeping Matter

The software development community is abuzz because of this article written by Tom DeMarco, author of that seminal work “Peopleware” that I ordered from some joker on eBay about six weeks ago but has not come yet.

You really should read the article.  I say this because I assume that you are a software engineer, like me.  Otherwise, let me summarize it for you.

Nah, that’ll take too long.  Let me sum up.

DeMarco basically says in the article the best software development on the most valuable software products with the highest degree of utility happens in the absence of control and measurement.  In other words, everything every software engineer on earth was taught about software engineering in college is wrong.

Yeah, that’s pretty much what he’s saying.

This has led to a lot of discussion around what we call all those people that we used to call software engineers just last month.  The impetus is that if the best software is created by not managing or quantifying the work and the deliverables in the project, is this really engineering per se?

Well, I’ve known this for a long time.  I just haven’t said anything about it.  Seriously, stop laughing.  I don’t expect you to believe me.  In fact, I’m happy to give credit to all the other people.  But it is certainly true that, although I’ve had that word in my job title for well over a decade now, I’ve felt for some time that software engineers are unique from other types of engineers.  Indeed, good “software engineering” requires creativity and artistry as much as it requires engineering.  “Software craftsmanship” is the term Jeff Atwood at Coding Horror is pushing today; “Software Artisan” is another term I’ve heard today as the new replacement for “Software Engineer.”

Whatever.  I know what it is I do, so you can call me a Software Masseur if you feel like you must.  I’ll just keep doing my job like always.

Joking aside, this article is a big deal because someone (more specifically, someone people will listen to) has finally come out and said what we’ve felt all along – trying to measure and control the creation of software is not helping, primarily because it is at least as much a creative process as it is a technical one.  Convincing the Employees Formerly Known As Software Engineers around the world this is true is not going to be a hard sell.

Convincing their employers and customers, on the other hand, is going to be tough.  That’s because the customers have come to expect that they will know what features will be in what products by what date, and they aren’t likely to give that up willingly.  They need (or think they need) to know all of this so they can plan – when to buy their next computer, when to get trained on the new version of a product, when to rollout system updates to an enterprise, when a partner can promise the delivery of new functionality to their customers, etc.

Interestingly, some organizations are able to avoid this expectation of their customers.  Open source projects are particularly good at this.  Eclipse, for example, delivers a release every year but makes no promise as to what features will be included in that release; other projects, like Linux and Apache, release whenever they feel like it.

I remember writing a software product that was going to be delivered as Apache modules for Apache 2.0, way back before Apache 2.0 was officially released.  We e-mailed the Apache project during development to ask when Apache 2.0 was scheduled to ship, and they said in essence, “We don’t know.  And, we don’t publicize or even set release date goals.  We ship it when it is ready.”  Of course that made it hard to plan, but part of me secretly wished I could get paid to work on Apache.

Open source projects are not the only ones that do this, however.  I defy you to figure out when any given Google app will ship, for example.  And consider Blizzard Software, one of the most successful video game companies EVER (ever heard of World of Warcraft?).  As I understand it, they are like Apache in this regard – their product will ship when it is ready, and not a day before, and it doesn’t matter what big holiday or trade show is coming up.  And so far they’ve done alright.

And then there’s Duke Nukem Forever.  Wait…

Setting up a new computer is one of those things that should be enjoyable, but is mostly just tedium.  That’s because there really isn’t a single OS out there that does for me everything I want in a single distribution – at least not one I’ve found.  In truth Linux comes closest, but in the case of Linux, there are still some things (like Motocross Madness 2, one of the best PC games ever) that you just don’t get there.

And don’t start giving me lectures on Mac.  Same problem applies there.  Even without games, I still have pretty much the same setup overhead for Mac as anything else.  Macs are great, don’t get me wrong, but I don’t write Mac software anymore, so I don’t have to be showing the Steve-love for a while now, until I start doing Mac development again.

Anyway, I’m willing to bet that at my new job my development machine will be a Windows machine – it’s just a hunch I have.  So here’s the rundown of setting the machine up for usefulness and effectiveness.

Basics
	First is Firefox. Firefox is a great web browser, fast and pretty reliable.  Once you’ve got Firefox installed, you’ll want to grab a handful of Firefox plugins.  When I set up next, I’ll be trying Google Gears, AdBlock, FireBug, Better GMail/GCal/GReader, Tab Mix Plus, FaviconizeTab, Fission, and GreaseMonkey of course.  I keep IE around because sometimes I need it, but I make Firefox my default browser.
	In addition to e-mail, I use calendaring and news readers (RSS/Atom) almost every day. A lack of decent free options in the past got me used to using Google for all of this stuff. But if I were to decide to use rich applications for these purposes instead, I’d give Thunderbird, Sunbird, and RSSOwl a try.
	Next is OpenOffice.org. I know, most people use Microsoft Office.  I realize it is better.  I realize it is more powerful.  I realize it is more ubiquitous.  It is also expensive for my purposes.  Stick with Office if you like it.
	For instant messaging I use Pidgin. Since I’ve got friends using MSN/Hotmail, Google, and Yahoo! among others, Pidgin gives me a great way to be able to chat with all of them in a single IM client.  And it has some great plugins that I’m eager to try out.
Multimedia
	For listening to and organizing music, I’ve been hearing a lot about Songbird and I think I’ll give that a try.  I don’t buy music from iTunes and I don’t have an iPod (I know, lame).  If I did I’d go with iTunes.  Although, I do like iTunes Genius feature, so I might go with iTunes just for that.
	When it comes to audio editing, Audacity is where it’s at.  I’ve used Audacity to make ringtones from some of my music MP3s, to edit and mix recorded WAV files into MP3 files, and even for my son’s science project to examine the differences between sound waves.  A must-have.
	In doing research for this blog post, I ran across these apps:  MediaCoder for translating and saving media files, Handbrake for ripping copies of your DVDs to formats for your handheld, and ImgBurn for creating DVDs.  So I haven’t actually used them yet, but I can hardly wait to try them out.  Managing video files and recordings is something I built my computer to do, but finding the software to get the job done has been tough.  Hopefully I’ll find the answer among these tools.
	DoubleTwist is a new application I’m eager to try for managing the transfer and synchronization of files from the PC to your handheld device.  I’ve got a really cool little Sony Ericsson phone that is supposed to work flawlessly with DoubleTwist; can’t wait to find out.
	I’ve used The Gimp for my photo editing for years and, for me, it removes any need for me to buy Photoshop.  I’m sure Photoshop users would disagree.  But hey, I’m not a graphic artist.  I’m just a guy who needs to edit photos from time to time, even for my job, and can’t justify the expense of Photoshop.  Lately, friends have been telling me about Paint.NET, and what they are telling me is that they like it better than The Gimp.  That’s a high standard in my opinion.  I’ll have to check it out.
	Ah – where would I be without Steam?  Steam’s client is free to download and serves as the launching pad for most of the games I play.  A lot of games I really like, such as Audiosurf and World of Goo, I first found out about via Steam.  Every time I launch it it seems there are more titles and more publishers available through Steam.
Security
	Truecrypt is a highly regarded application for encrypting data on your PC – one I’ve been meaning to try for a long time and plan to soon.  I know, I should do this.  Eraser, on the other hand, is one I’ve used for a long time.  It makes it really easy to truly erase files from your computer by performing multiple overwrite passes to keep your data from being restored after you’ve intentionally deleted it.
	For managing the applications that launch automatically when your PC starts, it is hard to beat Mike Lin’s Startup Monitor and Startup Control Panel applications.  Startup Monitor runs discretely in the background, and just notifies you when some application has requested to be run at startup, allowing you to decide whether to accept this or not.  Startup Control Panel offers a simple view of the applications already scheduled to run at startup, and allows you to disable them.  Great for improving boot times and free resources, not to mention helpful in keeping your desktop secure from rogue apps running in the background doing who knows what.
	Spybot Search & Destroy is essential for keeping your PC clear of spyware and adware that want to do evil things behind your back.  It integrates with most common web browsers, including Firefox, to help lock them down to avoid evil cookies and other tracking software from sending information about you to others.
	If you are up for trying a free PC antivirus application, ClamAV is the answer for you. Okay, I haven’t used it and probably won’t for a while because I’ve already got a license for a security suite.  But if you are in the market it is probably worth a try.  It certainly could not be worse than BitDefender (that steaming pile).
Utilities
	Daemon Tools is a simple utility that can mount local disk images as filesystems.  Mac does this very easily, of course, with .dmg files, but you need a tool like Daemon Tools to do it on a PC.  Use Alcohol 120% to create mountable disk images from game CDs, for example, which will enable you in most cases to play PC games by mounting the disk image in Daemon Tools instead of inserting the CD.  Or use it to mount ripped DVD ISOs when you are converting them to a format you can use on your handheld.
	Many years ago, Novell had this really great product called iFolder that you used to synchronize files between multiple computers.  Like many Novell products, it was a really awesome product that nobody ever heard about because Novell can’t figure out how to market anything.  But Novell employees know about iFolder and most of them are like me – once I got used to using it I could hardly stand to not have it. When I left Novell this was a big big problem for me. Finally Dropbox came along to address my problem.  Dropbox allows you to do what iFolder did years ago – synchronize files between multiple computers.  Dropbox is not nearly as full featured as the latest iFolder 3, but at least this one you can use without being a Novell employee.
	As I’ve said before, once I started working for Mozy I realized that online backup should be considered essential for anyone.  I really don’t know why a person wouldn’t use Mozy.  Even if you are backing your data up on a second drive, USB drive, thumb drive, etc. you should be using Mozy, to automatically provide a secure backup copy of your data in a separate location – for recovery from fire damage, for example. Having worked at Mozy for the past 14 months, I can vouch for their solid technology which is, in my opinion, the best in the industry without question.  You need online backup, so why not use Mozy?  You can back up 2GB for free or as much as you want for $5/month.
	For archiving and compressing infrequently-used data, I recommend 7Zip. It will unpackage almost anything and will package in the most common formats, including Linux-compatible TAR/GZ formats.  It’ll also do encryption and self-extracting packages in some formats.  In other words, it’s pretty much everything you want in an archiving tool, for no cost.
	I haven’t tried Everything yet but I plan to soon.  This is a highly rated desktop search engine along the lines of Spotlight for Mac.  Windows search I mostly use as last resort, but if this is anything like Spotlight I’ll use Everything all the time.
	Freemind is a note-taking application that I’m eager to try.  I’ve been needing one of these for some time, so I’m anxious to give it a shot.
	I consider Cygwin an essential PC utility.  Since I’m a lot more familiar with the Linux shell than the PC DOS-style shell, Cygwin provides me with a command prompt I’m comfortable with.  Cygwin comes with a large number of helpful tools, like the GNU C complier suite, ssh/scp, wget, and others.
	I’d also consider SQLite an essential PC utility.  SQLite is a very simple file-based SQL engine that is very useful and freaking awesome.  I’d recommend a PC utility for using SQLite but there really isn’t a good one.  Probably your best option is SQLiteSpy.
Application Development
	I’ve said before that if a person’s going to learn to write software, I think the two most important languages to learn are C and Python.  Since you already installed Cygwin you probably already have a C compiler on your machine, so now you need to get Python installed. In addition to Python, Ruby seems pretty interesting and one you should definitely look at, in addition to Python (and not instead of Python, not yet anyway). You should note that if you plan to do Python and/or Ruby development, you’ll probably want to get used to doing that natively on your machine, and not via Cygwin.  So don’t depend on the Cygwin Python and Ruby interpreters – use the native interpreters instead. If you must, use Java.  Sometimes there’s stuff you just can’t do without it.
	Ah, Eclipse:  the mother of all development environments.  Having worked closely with the Eclipse foundation and been part of starting an Eclipse project myself, I have a strong affinity for Eclipse.  For Java development, I’d consider it one of the best, if not the best, Java IDE available.  It’s also a great free alternative for a lot of other languages and application types.  Get not only the base Eclipse, but the plugins for C/C++, PHP, RCP/Plugin development, Data Tools, Test and Profiling Tools, and Web Tools. And if you aren’t going to get the Eclipse Python plugin, you’ll want to install Eric instead.  Eric is a pretty good little Python IDE that works on both Linux and Windows.  You’ll need PyQt for Eric to work I think. If you are really wanting to do development in C# and .NET instead, but don’t have the .NET platform, you could try SharpDevelop.  I haven’t tried it though, so I can’t say – and in my new job, I’ll be doing my C# development in Visual Studio, which is certainly better.
	If you want to try out simple GUI programming, especially cross-platform GUI programming, try wxWidgets.  You can program directly to wxWidgets in C or C++, or in Python using wxPython.  If you think GUI programming with wxPython is your cup of tea, you might also want to try Boa Constructor, which is a good Python development environment with GUI building tools.
	For web development on a Windows PC, I love WAMP.  This simple bundle offers Apache, MySQL, and PHP all together in a single package that you can easily start and stop all as one. (This is the part where the Mozy PHP bigots comment to tell me how rotten PHP is, and where they tell me how much better Perl is, and where I nod and pretend to agree in order to keep the peace.  So bring it on.) WAMP is great for your typical free-style web application development, especially if you are building from an existing framework, which is quite likely to be built in PHP.  If you’re building from scratch, however, you would probably want to strongly consider Ruby on Rails, in which case you’ll want to install RubyGems to get Rails and other goodies. Finally, I hear Kompozer is a pretty good HTML-style editor and page builder, and I might give that a shot sometime.

Yesterday was my last day at Mozy.

I left Novell and came to Mozy just over a year ago.  For a variety of reasons, I chose to leave Novell even though in my case I had to take a cut in pay to do it.  I don’t regret this decision at all.  I’ve learned tremendously from Mozy, met a lot of really great people, and really loved working at Mozy.  It’s a great company with a great product, but the best part about Mozy is the people – there are some really great people there, many of which I didn’t really get to work with as much as I would have liked, and I’ll definitely miss the opportunity to work with them.

At my one year mark last May, I took a step back to look objectively at my job at Mozy.  This happened to coincide quite closely with EMC’s recently announced employee 5% pay cut, which of course also affected then-current Mozy employees.  (By the way, you have to love how EMC told the press that they were “asking” employees to take the cut – it was more like “we’d like to ask you to take this cut in pay, or leave.”)  I looked at the tradeoff I’d made a year earlier, now with greater insight, along with the adjusted compensation, and realized that I wouldn’t have made the decision to come to Mozy last year after all.

It’s an odd conclusion because I still would have made the decision last year, knowing everything I’d learn and the relationships I’d build.

It was at this point that a great colleague from a former team at Novell, one of the two best teams I’ve ever worked on (the other also from Novell), told me that the company he is now with was hiring, and asked if I would be interested.

Truth is, he and I had discussed this exact position about a few months before, and I’d told him at the time that I was happy where I was.  But then EMC changed the dynamics of our employer/employee relationship, and when he approached me again, by this time I was willing to consider a change.  And as my new employer and I considered it, we both came to the conclusion that it would be a good thing.

I’m really looking forward to this new opportunity.  I’ll learn a lot, I hope to contribute a lot, and know for a fact that the team I’ll be on there is outstanding.  At the same time I’ll miss the cool of Mozy.  It’s still a great place with great people.  Especially if you are considering working there now – the 5% cut does not apply to new hires, and I know they are still looking for great engineers among other things.

Uh, this is a programming post.  Just so you know.

The Problem

Okay, so let’s start this discussion with a simple linked list that contains the ingredients for Bill Cosby’s Chocolate Cake for Breakfast:

Fig. 1 - Linked list of ingredients for Chocolate Cake for Breakfast

Normally we’d use a linked list in programming when we know we may have to arbitrarily insert items later on.  For example, I just realized that this cake does not contain chocolate, so I should be able to insert that into the list by shuffling a little bit:

Fig. 2 - Chocolate Cake for Breakfast, with Chocolate Added

This seems like a much tastier recipe.  But there can be a slight problem with this.

Suppose the linked list is serving as a model for your recipe view.  You have a UI that accessess the model and then displays the results in some sort of view that you can see on your laptop in the kitchen.  Suppose further that you decide to go even crazier and not just stop with adding chocolate, but you are also going to add sugar and frosting.  INSANE!

This is not a problem yet.  Likely your UI offers the ability to add new ingredients through the UI, so you just add them and the model gets updated, and when your view refreshes you see the new list:

Fig. 3 - Chocolate Cake for Breakfast - Dessert Style!

That sounds like a great chocolate cake.  But now suppose Mr. Cosby is on stage in Winnemucca doing his Chocolate Cake for Breakfast routine, and for some reason that nobody can understand the teleprompter that reminds him of the routine is using your data model to render the view – right at the exact moment in time that you are changing the recipe.

Uh-oh.

I’ve run into this problem a number of times in my software development career, where I have a data structure that is being accessed by more than one user at the same time.  In our case, we have one read-only accessor and one read/write accessor.  At first glance it doesn’t seem like this should be a problem.  But unfortunately, even though the reader isn’t making changes to the model, the reader still depends on the model having a constant state.

There’s a couple of fairly simple ways to address this problem.

First, we can make a copy of the model for each user.  We can have a resource that provides a copy when a user requests the copy.  Then that user can do whatever they want with the model – read or write.  When they are done, they can either discard their copy, or turn it back to the resource for the changes to be merged into the whole.

This might work for some implementations.  In our trivial example here it would probably be just fine.  But if I have a data structure with two million 50-byte records, this isn’t such a great idea.  It isn’t just that I don’t want to make a copy of a 100 Mbyte (okay, 95 for you technicality folk) data structure – I don’t want to double my memory usage from 100 Mbytes to 200 Mbytes (or 95 to 190, sheesh).  So making a copy won’t always work.

Another option is to lock the structure.  Using some sort of a mutual exclusion primitive I can lock the data structure so that only one user can access it at a time.  I can’t just hold the lock per-access, though – I have to maintain sessions and locks for the entire session.  In other words, I have to make sure that nobody can change the recipe at all throughout the duration of Mr. Cosby’s act.  This high level of granularity in the locking of the data structure makes it hard to use.  And it is particularly frustrating for read-only users who are thinking, “Why can’t I access the structure?  All I want to know is what is in it!”

Versioning

One way I’ve successfully solved this problem in production code is to use a versioned container and versioned iterators for that container.  I used these at Volera, for example, for some reason that doesn’t matter now, because Volera is DEAD.  Anyway, here’s how they work:

Versioned Containers act pretty much like regular containers – vectors, lists, hashtables, etc.  The difference is that each link from one element in the container to the next has an associated version number on the link.  Any given element can have a number of “next” links – even a singly linked list, like in our example.  Each link would have a different version number.  The container itself also has a version number, which corresponds to the highest numbered link in the container.

Versioned Iterators go along with the versioned container.  When you request an iterator from a container, it gives the iterator a version number which corresponds to the current version number of the container at the time of request.  The user requesting the iterator can also request a different version number.

Here’s what our data structure might look like if it were versioned:

Fig. 4 - Chocolate Cake for Breakfast with Versioned Containers and Iterators

This ends up working out pretty cool.  Mr. Cosby can request a versioned iterator of the view when his show begins that displays the model as it is when his show starts.  Let’s say that his iterator is version 1.  That iterator traverses the list by only following links where the version number is less than or equal to the version number on the iterator.  So it first goes to “eggs”, then follows link 1 to “milk”, then follows link 1 to “wheat”, then ends.  It can be doing this at the same time that another user is changing the data structure, first by inserting “chocolate,” which creates the version 2 link from “milk” to “chocolate” and the version 2 link from “chocolate” to “wheat,” and later by inserting “sugar” and “frosting” as version 3.  A version two iterator on the structure would start with “eggs” then follow the version 1 link (remember – less than or equal to the iterator version) to “milk.”  But then it would follow the version 2 link instead of the version 1 link, so now it goes to “chocolate” instead of going directly to “wheat.”

The same goes for the version 3 iterator, which gives us everything in the data structure.

There’s a couple of issues you have to deal with when you implement a structure like this.  First off, you need some sort of a scheme to go through the structure and prune it of old traversal versions.  If I remember correctly, a scheme that will work to do this is to have the container itself remember how many users are accessing it and what versions they are at.  When one iterator goes away, the iterator should report this back to the container.  The container then looks to see if that iterator is the lowest-version-number iterator that it knew about.  If it is, the container can go through and prune all of the links that are versioned at that number and below, as well as nodes in the structure that are only included in the structure by links at that version and below.  So there’s a little bit of housekeeping that has to go on, but the container should be able to do this after iterators go away.

In other words, once Mr. Cosby’s show ends, the data structure that previously looked like Fig. 4 should end up looking more like Fig. 3.

Another problem to consider is this:  If the data structure lives for a long time and has lots of accessors, it is possible for the version numbers to overflow.  The overflowing isn’t necessarily a problem, though, as long as you keep track of what is really the oldest version in the data structure.  Another way to do this is, as you prune the table, you can take the liberty of reversioning the table if there are no accessors to the table.  Depending on your situation, you may never have a chance to lock the whole structure in order to reversion it, but the first option should be workable.

Lastly, since you have to version the links themselves, you might not be able to directly use standard containers; for example, if you are using C++ you might have to create your own linked list template with a list of versioned next pointers in each node, instead of implementing your versioned list container in terms of the STL list container.  I never tried this so I’m not sure how it would work.  When I did this, I did it using a tree container that was implemented in STL style, with versioning in the pointers and iterators.  It was pretty cool.

I went to the doctor again today to see him about my back. I did not tell you about my back before, because frankly my personal health is none of your business, Mr. Internet.

I went to see him again, because the first time I saw him he told me if it was still hurting in two weeks to call and make another appointment.  So after two weeks when it was still hurting I called to make an appointment and they said, “Oh, he is on vacation.”  So I waited two more weeks to see him today at 10:00.

HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA!!!!!!!

Actually, that is when I was scheduled to begin sitting in the public waiting room.  Once I checked in they informed me that it was about a one hour wait, which in retrospect was incredibly optimistic, percentage-wise, as I waited over 90 minutes for my name to be called.

Of course, that was the amount of time you had to wait to go to the private waiting room, where the wait times are only about 1/2 as long.  I waited there for another 45 minutes for the doctor to come in.  I will now quote for you the entire session.  Okay, not really.  But pretty much.  Okay, here goes:

Doctor:  So, your back still hurts?

Me:  Yep.

Doctor:  How long has it been?

Me:  6 1/2 weeks.

Doctor:  Correct.

(That was the quiz part, to see if I’m really the correct patient.)

Doctor:  Okay.  I think you should go to physical therapy now.  Here is a referral.

There is no possible way my actual appointment lasted longer than five minutes.

But hey.  Why get mad?!  I decided before I even got there that I was not going to be one of those people that complains about the wait time, because who cares?  Not the receptionist!  So instead I said to myself, “What can I learn from this?”

And then it hit me – I can apply this to my own profession!  And so here you have it – the MDMethod for Software Development Estimation ©.

It is mostly like other kinds of software development estimation, which is to say, some combination of voodoo, magic, and recently used hay.  Here are the key points:

• There is no correlation between when something could be delivered, when it should be delivered, and when it will be delivered.
• Feel free to ignore reality and set unrealistic expectations.  Use an attractive woman to do this; her clothes should show cleavage and she should sit at about mid-torso level of a standing average-height man.  Teach her the expression “Blinking Bambi Eyes” – this should be used once expectations have not been met, at which point she simply says, “Yeah.  I’m really sorry about that!” in a sympathetic voice that seems sincere.
• Actual delivery of value can be offered within a confidence interval of reasonable estimate time plus a fudge factor of about 3000%.  The doctor in my case beat the margin of error by about 300% – it took 135 minutes to deliver to me my five minute appointment, which is a margin of error of 27 times or 2700%.

Here’s how I might do this in practice:

• Since I am not an attractive woman, I hire an attractive woman to set unrealistic expectations for me.  My wife is the best candidate for this job, but I respect her too much, so I will have to settle for another less-than-ideal candidate.  From this point on I will refer to this person as “Janice,” which seems like a good receptionist name.
• Let’s assume that you represent the company.  You ask me to deliver a feature.  I tell Janice to inform you that I could get started on that in two weeks and that it should take about a week to implement.
• When two weeks pass and you ask Janice how I am doing, she says, “Oh, he is on vacation.  He will be back in two weeks.”  When you say, “Wow, that is upsetting, because he said he could start on this in two weeks, two weeks ago,” Janice will then deploy Blinking Bambi Eyes and say, “Yeah, I’m really sorry about that!” in a sympathetic voice that seems sincere.
• In two weeks when you come back and asks Janice how I am doing, she says, “Yes, he is starting on that today.  The wait time for that is three months.  We will let you know when he is ready!”  She says this in a cheerful and playful tone and does not necessarily mind if you happen to notice her cleavage.  When you say, “Wow, that seems like a long time to wait; I wish we had known it would take this long earlier,” Janice again deploys Blinking Bambi Eyes and says, “Yeah, I’m really sorry about that!” in a sympathetic voice that seems sincere.
• After you wait for four months and I still have not started, you go to see Janice, who does not try to conceal her cleavage while you express your dissatisfaction at the fact that you are still waiting for me to start on your project.  At this point the initial two week wait doesn’t seem so bad now does it!  Anyway, Janice again deploys Blinking Bambi Eyes and says, “Yeah, I’m really sorry about that!” in a sympathetic voice that seems sincere.  By now you are just a little bit irritated – not really mad, just slightly irritated – because you have been sitting in the waiting room for four months.  But how can you be angry at Janice?  The face!  The Blinking Bambi Eyes!  The cleavage!  It is not her fault anyway!  So you just keep waiting.
• Finally, after four and a half months, Janice happily informs you, “Matt is nearly prepared to start thinking about when he might be able to begin working on your project now!”  This excites you so greatly that you forget all the time you’ve waited.
• Six weeks go by, but this is nothing.  You are surely on the home stretch.  Finally, I show up.  I say, “Hey, you know that project you asked me to do?”  You say, “Yes.”  I say, “When did you ask me to do that?”  You say, “Six months ago.”  I nod my head because you have passed the test – you are indeed the person who asked me to do this project.  Then I say, “Well, I did it.”  And then I leave.
• You are thrilled and happy to pay lots of money for my professionalism and high quality of service.

This seems like such a good idea.  I am going to start using this technique at Mozy right away.  I will follow up with all of my success stories so you know how well it is working.

P.S.  I was just kidding about the cleavage.  I am happily married.  I did not notice all of that cleavage.  At all.

My inbox is overflowing with emails from people asking, “Matt, how can I be more like you?”

Honestly, that is only sort of true.  Some of the emails say, “Lose 30 pounds in 12 minutes!”  And some say, “I am the prince of Nigeria!”  And some say, “I am lonely and looking for a purely physical relationship!”

More honestly, that pretty much comprises the bulk of the email I receive.  Nobody has actually asked how to be more like me, strangely enough.  But that day is surely coming, and since I am a computer scientist, or at least since USU says so, I could start with a list of books that you can read if you want to become a true nerd and rule the world the way I do, which is to say, not.

(So, in fairness, I haven’t actually read all of these or owned them. There are some that I haven’t read, but I read one like it; those are marked in blue. There are some that I haven’t read but think I should; those are marked in gray. There are some that where I read one like it, but I want to read that particular one – they are bluish-gray. And I didn’t list the probably 40-50 CS books I own or have owned and read that are not shown here. So cut me a break.)

Math and English

First off, in order to be a good software engineer and computer scientist, you have to be a good mathematician and a good writer.  Sorry.  You simply can’t be a competent software engineer without a solid mathematical background, and you can’t be an effective one if you can’t figure out how to express your ideas clearly in writing.

Learning to Program

Your next step is to learn basic programming concepts.  In my opinion, you should learn two languages at this point:  Python and C.  Python is a good beginning language, very easy to create real applications, easy to learn, and very versatile and useful in the real world.  C is the fundamental systems programming language.  Knowing Python and C will allow you to program just about anything and gives you a good fundamental background.

Computer Science Fundamentals

Having learned how to write basic computer programs, now is time to get into the science of computer science.

Programming Technique and Methodology

How to write software well.

General Programming

Two other languages you might want to know are C++ and Java.  C++ is much maligned, but widely used, especially for systems applications, games, and other high-performing software applications.  Java is an abomination in every sense of the word.  But it is also very popular and good to know.  If you are going to learn Java, you should also learn JNI, so you can get from Java back to C and get some real work done.

Systems Programming

If you are going to do systems programming, you’ve got to know the specifics of how to program to the environment in question.  It’s worth noting here that the UNIX books basically cover POSIX, which applies not only to UNIX but BSD, Linux, and Mac as well to varying reasonable degrees.  I’ve also included an internals book for the big three platforms (Windows, Linux, and Mac).  And if you are going to program for Mac, you will probably want to learn another language: Objective-C.

UNIX/Linux
Windows
Mac OS X

Other

Every good software engineer should clearly understand open source; hence The Cathedral and the Bazaar. You will find you are missing out on a number of inside jokes if you don’t read The Hitchhiker’s Guide to the Galaxy. And everyone should read The Code Book, simply because it is so interesting.

Image Credits: amazon.com and barnesandnoble.com

And by that I mean, the interaction is dumb, not the user.  Since I was the user, I think that goes without saying.

Anyway, earlier this week AMEX sent me a survey request, since I used their tool at work to book travel to Seattle last week.  And since my back kept me awake all night before, I had a low tolerance for dumbness.  So I got this survey, and I thought I would share the answers.

Moral:  Don’t ask dumb questions.

I hope it helps them.

The movie “French Kiss” with Kevin Kline and that one chick is one of me and Amber’s favorite VHS movies.  (In our house, there are three time periods to movies – DVD movies, VHS movies, and pre-VHS movies.  No Blu-Ray yet.)  I like it, but the reason is not because it is a romantic comedy with that one chick in it.  And the reason is not because it says “French” in the title and I’m trying to suck up to my boss, Luis, who lives in France.  No, I like it because it has some really, really funny Canadians in it.  Like Strange Brew!

No.  Actually, not.  But it is a pretty funny movie.  And there is this part, which is not funny, where Luc is explaining to that one chick about what makes wines different from each other – that they take in elements from their environment that contribute to their unique flavor and texture.  For example, if you spill some wine in the dirt, and then scoop it up with your cup and drink it anyway, you will probably get some dirt crumbs in there, which changes the texture from “smooth” to “gritty” and just tends to get you even more drunk than you were before.  And that’s why I don’t drink.

Anyway, I was thinking about that today, and how similar that is to software engineers.  See, software engineers, also, take in elements from their environment that make them unique.  In fact, an easy way to say this is that software engineers who live in one part of the world are better than software engineers who live in another part of the world because of their superior geography.

For example, you might have one software engineer, let’s call him Steve.  He might live in an peninsula that is called a valley, surrounded on three sides by ocean, with real estate that is unreasonably expensive.  Or you might have another software engineer, let’s call him Bill, who lives in a place where it rains all the time, in the land of Nirvana and Alice In Chains and Soundgarden and Pearl Jam and Starbucks.  Or another guy, named Nathan, who lives in an area with tight roads that wind all over creation, also with expensive real estate, elitist professional sports teams, and where nobody ever pronounces the letter “r”.

Compare these people to someone named Drew, for the sake of argument.  This guy, also a software engineer, lives in an area with big mountains, lots of Mormons, and secret sand dunes where someone exactly like myself can go riding motocross bikes.

Anyone can see that there is simply no way Drew will ever be able to measure up to the likes of Steve, Bill, and Nathan.  I mean, just take into account the geographic considerations!  How can big mountains ever hope to make you the kind of software engineer you could have been if instead you had been surrounded by ocean or Soundgarden?  The simple answer is:  they can’t.  I mean, be serious.  This, my friends, is the effect of geography on software engineers – geography can make the difference between you being really excellent and simply mediocre.  Secret sand dunes are really amazing, but they obviously cannot make you into the kind of software engineer you could have been if instead you were to replace those dunes with expensive real estate.

I manage the Mozy Windows Client engineering team, and not long ago I sent out a meeting request that began thusly:

Guys,
We shipped 1.12, which means we are awesome, and shipping 1.12 and being awesome is something worth celebrating.

Now, before all you fairer readers get offended, I’m allowed to address my local team with “Guys” because all of them happen to be men, which is NOT the case with my extended team (hey Seattle peeps).

Anyway, I took my team to the movie today to celebrate releasing Mozy 1.12, and being awesome.  Most of my team chose to see “Taken,” but Pancho preferred to see “Race to Witch Mountain,” and since I didn’t care and didn’t want Pancho to be alone, I went to see that movie with him.

And it wasn’t bad.  For one thing, it has The Rock in it.  The Rock’s screen name is Dwayne Johnson, and he is one of the greatest actors ever, where “greatest” means “biggest and strongest.”  In case he reads this blog, let it be known:  The Rock, you RULE!  Freaking RULE!  Please don’t beat me up!

However, I hadn’t realized that you would have to know so much about the computer game Starcraft in order to understand “Race to Witch Mountain.”  And this may also make the movie more geeky and make more geeky people want to watch it.  Plus, probably a lot of them will wonder why the movie seems to be poking fun at the space alien convention – but that is another story.

Anyway, here’s the lowdown:

• Sara and Seth are aliens from outer space.  But they don’t look like aliens.  This means they are obviously Terrans.
• When everyone consults with a space alien expert, he describes aliens as “praying-mantis-like.”  It is apparent that he is only familiar with Zerg aliens, and assumes that since Zerg are aliens, therefore all aliens are Zerg.  Which is a reasonable assumption to make, although anyone familiar with Starcraft knows how false this is.
• There is also another alien.  Now I don’t mean to spoil the movie, so let me just say he’s an assassin trying to kill Sara and Seth, and that is pretty much the whole plot.  I probably spoiled it.  Eh.  Anyway, by the end of the movie it is quite clear that this alien is a Protoss.

So if you are thinking about going to see “Race to Witch Mountain” (which you should, so The Rock doesn’t hunt you down and beat on you), you should read about Starcraft first – and then you’ll be ready to fully enjoy the movie.

Oh, and by the way – if you know C and C++ and are awesome, you should talk to me about working at Mozy.

Mozy is hiring.  I mean, Decho is hiring.  Decho is the silly name given to replace the awesome of Mozy.  We still call it Mozy, we can’t help it.

Anyway, we’re hiring.  Specifically, my team, the client team, is hiring.  And since Mozy decided to make me the manager of the Windows client team, that means I’m participating in the interviews.  This is stressing me out, because I feel like I am deciding the fate of people.

The process of getting hired at Mozy (arrgh, Decho) goes something like this:

• Apply and submit a resume.
• If we like your resume we will do a phone interview.
• If you do well in the phone interview we will bring you in for on-site interviews.
• If you do well in the on-site interviews we will assign you a homework assignment.
• If you do well on the homework and you are the best candidate we have for the position, there’s a decent chance you’ll get an offer.

Development work at Mozy is primarily done in C++.  Objective-C on the Mac side.  Pretty much you have to know C and C++ and/or Objective-C to get a development job, unless you want to work for the web team, using Ruby.  But those guys are kinda weird.  They sit on a different floor in the building and everything.  We’re not talking about those guys.

So during the phone screen, we ask you to rate yourself on C++.

We explain the rating scale like this:  0 means you are my father, waiting for this computer fad to go away, and you haven’t really heard of C++.  1 means you wrote Hello World in C++ once, and might be able to do it again today.  On the other end of the scale, 10 means your name is Bjarne Stroustrup, or maybe Herb Sutter or Andrei Alexandrescu.  9 means you have written books on C++; 8 means you could write a book on it, or teach courses on it.

Please, people.  Do not flatter yourself on the C++ scale.

I interviewed with Google once, over the phone.  They asked me this same question with pretty much the same scale, except they made no mention of my father.  I told them I was a 6 or a 7.  And I actually have taught courses on C++.

Lately we’re asking people this question and invariably we’re getting people saying, “Oh, based on that scale, I’m a 7 or an 8.”  Even kids in college.  Now I’m not saying that kids in college can’t be a 7 or an 8 – just, keep in mind, we’re not seeing a lot of true 7′s or 8′s among experienced professionals.  I’m just sayin’.

When you say in your interview, “I’m a 7 or an 8,” what you are telling me is this:  “I know C++ better than you.”  Now, you don’t probably know me personally, so hey, maybe you are better.  All I’m saying is, you’d better be ready to prove it when we bring you on-site.

For example, you’d better know at least most of this stuff:

• How to define a template class
• How to correctly define the assignment operator for a class
• How to overload the insertion and extraction operators for a class you define
• How to iterate over an STL vector
• Whether ++i is better, worse, or the same as i++, performance-wise, and why
• What methods the compiler will create for you if you don’t create them yourself, and the implications of this
• How to indicate in your developer contract whether a class is meant to be subclassed, which methods are overrideable, and how you insist that only subclasses can be instantiated
• How to specify default values for parameters

If you are a 7 or an 8, you probably should have read most of “The C++ Programming Language” and/or “Effective C++” and/or “Advanced C++” and/or a number of equivalents.  Having read “Design Patterns” would certainly help, although lately those have kinda lost their glimmer and so I don’t weigh on those like I used to.

Also:

• What const and mutable mean

Yeah.  const.  Don’t be like the self-proclaimed C++ expert I worked with at Enterasys Networks, who told the whole company he was the go-to guy for C++ questions, who, when asked, “Why does it say const after this method declaration?” replied, “Oh, they just do that a lot in C++; it doesn’t mean anything.”  Yeah.  Don’t be a doofus.

Don’t try to impress me by saying you are a 7 or an 8 if you aren’t.  Really – you don’t have to be a 7 or an 8 to get a job at Mozy (Decho…hrm).  If you say, “Oh, I’m probably a 5,” that tells me you are a good, solid C++ dude (or dudette, whatever) that knows how to write decent C++ applications.  You’ll probably get asked to come in for an interview anyway.

When we bring you in, it is my job (and Cody’s) to figure out how much of C++ you really know.  We will start out at the point you specified and go from there.  If you are really a 5 or a 6, but you said 7 or 8, you will feel like we’re being very brutal on you.  Hey, you are the one who said you knew your stuff.

Oh – one more thing.  Some of you experienced hires don’t think you should have to go through all of this to get a job with us.  Well, we make the rules.  Every one of us that works there has gone through it.  If you think the rules of Monopoly are dumb, nobody’s gonna think bad of you if you decide not to play.  But if you want to play but not follow the rules, well, don’t be too surprised if people take issue with that.  If you’re gonna try to work at Moz – uh, Decho, at least for my team, just go through the process like everyone else.

Okay, I feel better.  Whew.  Oh, and by the way, if you really are a 7 or an 8 (or better), I have a link for you.