No Source, No Docs, No Joy

I recently worked on a couple of stories for a project which uses a certain commercial C++ library in its implementation. This library is truly a magical black box; it has a fairly simple API but behavior that is unintuitive and complex. The documentation is minimal, and the product is closed-source (and decompiling or stepping through a disassembly is a rather painful and time-consuming way to understand how something works). The company's support was okay, but less-than-responsive. And there is basically no community around the product. Because of this, working with the product was a slow, frustrating grind. Stories that "should have" taken about two weeks to finish ended up taking over three weeks entirely because of how difficult this library was to use.


The next story I worked on involved our persistence layer, which uses MongoDB. In contrast to the (nameless-to-protect-the-guilty) C++ library, MongoDB was for the most part a pleasure to use. It's a much larger product, and much more complex; we're a good ways from really understanding best practices. But the documentation is good and there is a healthy community around the product, so it's easy to get help.

Good products make it easy to overcome the initial learning curve and to answer "how do I..." questions with at least two of: good documentation, source code availability, good support, and a healthy community. These products are enjoyable to use, and any member of the team can almost immediately be productive when using them. In contrast, when some hard-to-use product is involved, one person often gets "stuck" with the story because they have the most experience with it. "Yeah I know we should spread out the knowledge, but Joe's the only one who knows how to use the FizzCrap API...". Poor Joe ends up spending half his time working with FizzCrap, and eventually he either goes postal and deletes the entire git repository or doesn't come in one Monday morning because he's found a job elsewhere as a night watchman.

Don't be the team that has a "Joe" on it. Usability should be one of your primary considerations when evaluating a product/framework/library, not an afterthought.

Quick Change Directory for Bash under Cygwin

I think about half the commands I type in a Bash shell must start with "cd ....". Since the Windows filesystem is inherently broken by design (http://secretgeek.net/ex_ms.asp) and pathnames can get pretty long anyway, this involves a lot of repetitive over-and-over recurring again and again typing of the same identical equal paths.

Doing the same thing more than once often presents an opportunity for automation. (See if automation will save you time first. Also consider whether your effort to automate something will help others. Then go ahead and automate it anyway, because you know you want to.)

Thinking I must not be the first person to have this thought, I did a bit of Googling and quickly found autojump.

Basically, autojump just remembers every directory you "cd" to, and you can then quickly jump to it. E.g. "j aut" instead of "cd /c/Users/Thomas/Projects/autojump". Five characters beats the heck out of 37.

Pluses: It just works, with very little cognitive overhead. After a few days of use, my fingers and brain are already starting to build a "muscle memory" of a few directories. I don't have to think about typing "j ..."; it just happens.

Minuses: It doesn't handle paths with spaces. There'll be no jumping to /c/Program Files/.... There's also a tiny-but-noticeable delay when you "cd" to a directory; it takes a while to spin up the Python interpreter. (Yes, it requires Python, so you'll want to make sure you have that installed.)


Installation is simple; I cloned and installed according to the instructions (I used the "--global" option to install under /usr).

Recommended.

Cygwin SMTP and POP3/IMAP Mail Setup

The client I'm currently working for is an encrypted email provider. At times development/testing involves a lot of variations on sending and receiving email, and on occasion I also want to be able to see exactly what's being sent/received. We have some development mail servers, and most of our builds are deployed via a Jenkins build to a VMWare farm -- but sometimes it's a bit of a hassle to use these, especially if they aren't configured for what I need or if someone else is busy on the host I need.

So I thought, wouldn't it be nice if I could set up an SMTP mail relay and POP3 server locally, so that I can run tests and experiments.

The result of that thought was about an hour of Google-fu and a few experiments. In brief, what I arrived at was Exim and inetd+uw-ipop3d running in Cygwin. I'd write up the details, but that's already been done. In short, if you want to run your own personal mail server, set up Exim by following the instructions here and set up inetd+uw-ipop3d by following the instructions here.


This will give you a bare-bones setup suitable only for sending and receiving email on your computer. On the SMTP side there's no authentication at all; DO NOT do this on a computer reachable from the Internet. And it's not configured to send emails to the world. On the POP3/IMAP side, when you install uw-ipop3d the user the service runs as will be the only user you can log in with. Or to put that another way, the only person that can receive email through IMAP is that user. I have a cyg_server account which got set up my sshd I think, and am using that. You'll log in using the normal Windows credentials for that account. (Note you can look at /var/spool/mail to see the mail for any other user who you send email to.)

Addenda/Errata:

Ensure you have /etc/c-client.cf with the following:

I accept the risk
set disable-plaintext 0

Compiling Git on Cygwin

First off, if you aren't using Git for your version control needs, you should be. Well alright, that's a little strong -- but if you're still using a non-DVCS (aka centralized) source control system then check it out. Seriously.

Now, on to the main topic of this post. I'm currently doing as much as possible from the Cygwin environment, and so naturally I have installed the Git package and use that instead of the more common (and perfectly usable, BTW) msysGit project. Unfortunately the currently shipping Cygwin version of Git is 1.7.9. That's a few releases old, and it bit me a little when I was Googling to figure out how to set the upstream URL for an existing remote. (The syntax for that command has changed a few times.)

So I thought I'd have a go at compiling the latest sources for Git. It turned out to be quite easy, but if you follow the "recipe" here it might save you a bit of Googling:

1. Run the Cygwin setup utility and ensure you have the following packages installed:
• gcc
• autoconf
• curl
• libcurl-devel, required for http/https support (NOTE: this is an obsolete package, you will need to uncheck "Hide obsolete packages" when selecting packages to install)
• make
• libiconv
• python
• perl
• gettext
• gettext-devel
• you may need libiconv-devel and cygwin32-liviconv (see Marek's comment)
2. Get the source. Either download a zip archive or clone the Git sources with git clone https://github.com/git/git.git using some other version of Git. A couple of notes on that with respect to Git configuration if you do this:
1.  Before you clone, make sure your Git config has core.autocrlf=false, otherwise you'll end up with DOS style line endings that cause the build to break.
2. I also had a screwy http.sslcainfo property value; either unset it or point it to /usr/ssl/certs/ca-bundle.crt.
3. From your newly cloned git repo, run the following commands:

make configure
./configure
make
make install

You should be able to run git --version and see something more recent than 1.7.9. (I had to open a new Cygwin console before this exact command worked, but I could run /usr/local/bin/git.exe --version.)

That's all there is to it (see, I told you it was easy). You're now running the latest version of Git. Thanks to all the folks who work on both Cygwin and Git for making this so easy.

Correctness, Clarity, and Changeability

I have a strong personal drive is improve my productivity as a software developer. Consequently I do a fair bit of reading and thinking about how individuals and teams create good software. Since this is a baby blog I figure it might be a good idea to start by exploring some of my ideas about what "good software" looks like.

Like any other human endeavor, software development is kind of messy. We don't communicate well; we make mistakes every day; we are horrible at predicting the future; we make irrational and imperfect decisions. But we still manage to succeed, and occasionally we succeed brilliantly. Developers who are the most effective -- who succeed brilliantly -- have learned how to become really good at creating good software.

So what does good software look like, from the perspective of a software developer? There are a lot of "right" answers to this. One that I've come up with and like is in the title of this post, and yes, I came up with those three words because I like alliteration. Good software has the attributes of correctness, clarity, and changeability.

Correctness means satisfying the needs of the customer. If it doesn't do what the customer wants and needs, it isn't correct. In development-speak, this means the software satisfies the functional and non-functional requirements. Functional requirements are the 'what' of the software; the features I can use. Non-functional requirements are 'qualities' of the software; things like security and performance and robustness and availability and usability. Correctness is subjective, which is why there is often a lot of (sometimes heated) discussion among stakeholders about whether some program ought to have fewer bugs, or have more features, or be more usable, or be more responsive.

Clarity means the code, build scripts, shell scripts, database queries and scripts, internal documentation, configuration files, and other bits necessary to create a piece of software are as easy to understand as possible. The source should be self-documenting and well-organized, so that its behavior and purpose are obvious. Self-documenting code is obvious in its intent. It's characterized by meaningful names; by constants instead of magic numbers. Self-documenting code makes it easy to answer the question, "What does this code do?" Well-organized code makes it easy to answer the question, "Where is the code that ...?" The package (namespace) structure is clear; classes in each package are closely associated; structural and behavioral associations and dependencies are easy to find. The overall project structure makes it easy to find all of the code, scripts, configuration files, etc. that need to be changed in order to implement a feature.

Changeable means that the software is as easy to change as possible. Implementing new features and refactoring to support new requirements should require a minimum of effort. Good design and a solid test suite go hand in hand with changeable software. Good design makes it easy to make changes and still have the system compile. A good test suite makes it easy to make changes and still have the program work correctly. Use good design principles like DRY and GRASP and SOLID, and write good unit tests, and you'll have code that is easier to modify.

So how do we create software that is correct, clear, and changeable? The best answer I have found is in the principles of agile development. Agile development allows a team to be effective and productive over time. Or to put it another way, agile methods lead to software that is correct, clear, and changeable.

Configuring SSH to Disable Host Key Checking

I often use scp to copy files between Linux/Windows systems, and ssh to remotely log in. In our work environment, we frequently redeploy new virtual machine ISOs. This causes scp and ssh to complain because the host's key has changed. Following the steps at I found at Linux Commando, here is how to configure SSH to disable the standard check for keys. NOTE: This can prevent SSH from detecting a real man-in-the-middle attack, so use with caution. I have configured this to only work with internal IP's. There are two ways to do this. The once-only way is from the command line:

$ ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no

This tells ssh to use /dev/null for the known_hosts file, and to not use strict host key checking. For a more permanent way, put the following into the beginning of your ~/.ssh/config:

Host 192.168.*
StrictHostKeyChecking no
UserKnownHostsFile=/dev/null

(In my case 192.168.* refers to the internal network. Your environment may be different.)

This works on both Linux and Cygwin.