Superdevelopers at Facebook

General
Facebook gets unprecedented traffic. They don't want things to go down so they... release changes constantly, a few times a day! They use peer review and their immense audience to see how well their solutions work, then iterate. The other teams *support* engineering, rather than being gatekeepers.I actually did this at a medium-sized site many years ago. Push change to live, capture logs for five minutes while people beat it, then undo the change. It worked well.
Read More

ntop and RRD

DevOps
"ntop" conveniently displays web pages with charts and plots of your local network traffic. To record historical traffic, it uses Round Robin Database (rrd). The default ntop package enables the RRD plugin, but doesn't give itself permission to write out its data. Here's how to get the two to work:sudo apt-get install ntopsudo install -d /var/lib/ntopsudo install -o nobody -d /var/lib/ntop/interfacessudo install -o nobody -d /var/lib/ntop/rrdTo test, start "sudo ntop" (probably in "screen"), then open the web page http://localhost:3000To activate the RRD plugin, select Plugins / Round Robin / Describe, then click the red "no" to become a blue "yes." You'll see messages in the terminal where you started ntop.After a few moments, you can view RRD graphs. One is in Summary / Network Load.
Read More

Empirical Data behind Software Development Techniques

Dev
Our friends at Microsoft Research have published four papers testing--and measuring--development ideas. One big surprise was that a company's organization "can predict software failure-proneness with a precision and recall of 85 percent. " It's more important than using assertions or increasing test coverage!Other results: test coverage isn't a direct measure of software quality, and assertions are useful.I was glad to see some numbers behind what I've experienced, that Test Driven Development increases quality. "What the research team found was that the TDD teams produced code that was 60 to 90 percent better in terms of defect density than non-TDD teams. They also discovered that TDD teams took longer to complete their projects—15 to 35 percent longer." Hmm.Exploding Software-Engineering Myths
Read More

Fast Parallel Downloading (for apt-get) #2

Dev, DevOps
This is how to upgrade your Ubuntu system, downloading several packages in parallel. See previous article for details:cd /var/cache/apt/archives/apt-get -y --print-uris -u dselect-upgrade >debs.listegrep -o -e "http://[^']+" debs.list | xargs -l3 -P5 wget -nvapt-get -u dselect-upgrade
Read More

Ultimate Quality Development System

Dev
Our friends at Divmod have come up with an extremely clear method of rapidly delivering high quality software. The basic idea is you write a bug report, create a code branch, then check in the branch with a "review" tag. When someone reviews and okays your code, you merge into the trunk. This method has lots of benefits, including taking the drama out of peer review, and making it very easy to safely test on multiple platforms. https://opensource.com/article/18/2/uqds (via Twisted)
Read More

Cheap Network Measurement

Dev, DevOps
You can't control what you can't measure. I want a fast local wireless network, counting only from my workstation to the router. If I change settings or drivers, how can I tell if it's an improvement or not? How can I gauge my network speed, including driver and network settings?There are jillions of tools available*, but they mostly 1) require a specific client on the server, or 2) don't actually send traffic, they passively analyze what's there.Here's a solution: transmit a largish file to the server, where it will ignore the data. Measure the network bandwidth locally using 'cpipe' or another tool.Setup: enable SSH on the router, install "cpipe" locally.cat /boot/vmlinuz-* | cpipe -vt | ssh router 'cat >/dev/null'I get lines like this, with statistics on each of many chunks…
Read More

Server names in Dnsmasq

Dev, DevOps
My domain name service (DNS) is running on a router using the wonderful Tomato firmware. When a computer boots up, it sends its hostname (i.e. "sally") to the router, which records the hostname and returns the corresponding IP ("192.168.7.10"). This is all grand. But, what if different services require a computer to have multiple names? That is, when the Puppet service fires up on my lappytop, it looks for the hostname "puppet" by default. Given the above scheme, a computer has one and only one name; DNS doesn't know that "sally" is also known as "puppet". Here's how to fix that. The router is running Tomato, which uses Dnsmasq as the DNS service. It automatically figures out the normal hostname/IP mapping, for instance "sally" in the above example. We can…
Read More

Fast Parallel Downloading (for apt-get)

Dev, DevOps
I'm rebuilding a Ubuntu server. Normally apt-get downloads one file at a time, which can get dull when you're installing 598 files. I found the tool "apt-fast" which downloads one or two files quickly, by downloading with multiple streams per file. This is somewhat sketchy, as it requires installation of additional software, assumes the file gets spliced together correctly, and doesn't gracefully handle network problems.I have a solution: xargsXargs walks on water. It is incredibly useful. In a nutshell, it runs a single command on a list of files. I'll post a lot more later, but here's how to speed up apt-get:cd /var/cache/apt/archives/apt-get -y --print-uris install ubuntu-desktop^ > debs.listegrep -o -e "http://[^']+" debs.list | xargs -l3 -P5 wget -nvapt-get -y install ubuntu-desktop^Replace "ubuntu-desktop^" with whichever task or package you want.…
Read More

Agile Software Development

Dev
I suggest everyone run, not walk, to read Alistair Cockburn's "Agile Software Development (2nd edition)." He talks about how people learn, and how Agile turns software development into a "cooperative game", like rock climbing. That is, developers have different roles, and each role supports the others so the team can progress farther, producing more software features to deliver to the client.The author's notes on how people learn helped me immediately in my teaching practice. My classes are both more accessible (can help more people) and more focused (presenting more sophisticated knowledge.)(Note the 2nd ed is a bit different from the 1st, and so recommended.)
Read More