code: use Bash to control workflow of processes

Dev, DevOps
On Stack Overflow, someone asked how to control a workflow of processes.  The issue is that sometimes the procs will jam up, and should be restarted.  The following code does this.One technique to control an errant pipeline is to count the number of pipeline processes after a while. If they haven't finished, then zap them and restart.In this example code we use ask Bash for its process id (the $$ variable), then use pgrep to find all sub-processes of the script. That is, pgrep tells us all the workflow process IDs. If there are any, then we use the corresponding pkill command to squash them before a restart.In addition, we use the date command to output a pretty log file, showing what the system is doing at what time.full post…
Read More

prevent multiple copies of script running at same time

Dev, DevOps
Single-process servers (and cron jobs) are easier to write.  If a script fires up, then starts processing a job that another copy of the same script is working on, then bad things happen.The traditional solution to this is a bit of a hack.  Script fires up, writes its unique process id (PID) to a magic file somewhere.  When the script dies we delete the file.  This works somewhat, but fails if the script gets impolitely killed -- the "lock" file still exists but the associated script is no longer running.The following trick is useful. When the script fires up it registers itself with a global list of strings held by the OS. If the code can't register itself, it knows the program is already running, and should exit.  When a…
Read More

trivial Jenkins configuration

Dev, DevOps
Knowing when your software breaks is useful. If you catch subtle errors soon after the change, it's *much* easier to figure out which bit of code "optimization" broke things for the users.Another benefit is fixing "works for me" syndrome. Even if you're building software just for yourself, it's nice to have a 3rd party to verify that you didn't do silly things. Forgetting to check in South database changes is quite easy, but that simple omission will break everything.Jenkins is a cranky to set up, because it's a collection of moving parts. The best setup would be having Jenkins rebuild your tests every time GitHub detects a source code change.The following setup is not as geeky, but it's much, much easier: tell Jenkins copy local source files and test, every…
Read More

interesting database performance article

Dev, DevOps
Postgres doesn't support that much compression directly. These guys decided to take the performance hit of compressing/decompressing things at the filesystem layer... but it's actually *faster*. Turns out I/O bound stuff like the database, even if you spend more CPU on compression, there are multiplicative effects if the data is smaller.http://citusdata.com/blog/64-zfs-compression
Read More

Profiling Django

Dev, DevOps
Install django-extensionshttp://pythonhosted.org/django-extensions/runprofileserver.html1) sudo pip install django-extensions2) add 'django_extensions' to your app's INSTALLED_APPS list.Run server in profiling modepython  ./manage.py runprofileserver --prof-path=/tmp 8001Do a querytime curl -i http://localhost:8001/account/eventboard/update/Write little reporting moduleprof.py -- given laterRun report/tmp/account.eventboard.update.017442ms.1363891644.prof :         202963 function calls (197801 primitive calls) in 17.443 seconds   Ordered by: internal time, call count   List reduced from 2280 to 5 due to restriction    ncalls  tottime  percall  cumtime  percall filename:lineno(function)    26201   14.956    0.001   14.956    0.001 /usr/lib/python2.7/ssl.py:154(read)        2    0.725    0.363    0.725    0.363 /usr/lib/python2.7/ssl.py:301(do_handshake)      937    0.470    0.001   15.069    0.016 /usr/lib/python2.7/socket.py:406(readline)       10    0.318    0.032    0.318    0.032 /usr/lib/python2.7/socket.py:223(meth)        2    0.238    0.119    1.291    0.646 /usr/local/lib/python2.7/dist-packages/httplib2/__init__.py:982(connect)In this case,…
Read More

notes: Simple Chef with Vagrant

DevOps
(placeholder for Chef & Vagrant article)  install Vagrant and Chef in Ubuntu$ sudo apt-get install vagrant chef $ chef-client -vChef: 10.12.0install Knife$ knife configure$ edit ~/.chef/knife.rbcookbook_path [ './site-cookbooks' ] use Knife to create a Cookbook$ knife cookbook create beer** Creating cookbook beer** Creating README for cookbook: beer** Creating CHANGELOG for cookbook: beer** Creating metadata for cookbook: beerWhen the "beer" cookbook runs, it creates a file in /tmp$ edit site-cookbooks/beer/recipes/default.rbFile.open('/tmp/beer.txt', 'w') {|f| f.write('tastyn') }Tell Vagrant to use Chef, and to run "beer" recipe$ edit Vagrantfile  config.vm.provision :chef_solo do |chef|    chef.cookbooks_path = "site-cookbooks"     chef.add_recipe "beer"  endBoot the VM, thus running chef-client and our beer recipe$ vagrant upVerify it worked$ vagrant ssh -c 'cat /tmp/beer.txt'tasty
Read More

Ssh tunnels are great

Dev, DevOps
At work I'm connecting to multiple Munin statistics servers. On my lappytop I'm only running one server, so how do I get multiple sets of results?  Answer: create a tunnel to another server!The Munin protocol is extremely easy, send "fetch X" to get statistics on X.  In my example df=Disk File usage.  Here's how to get local information, via Munin running locally on port 4949.$ echo 'fetch df' | nc -q1 localhost 4949# munin node at freebeer_dev_sda1.value 5.29318865322941_dev.value 0.0339391645617528_dev_shm.value 0.378827479751794_var_run.value 0.00922469512382616_var_lock.value 0.Here's how to make a remote machine's Munin (on port 4949) show up on localhost (port 4950). This means we can scan multiple local ports to get information on many different machines.ssh -fNL localport/localhost/remoteport remotehostOption "-f" means drop into the background after asking for a password.  Next option "-N"…
Read More

ntop and RRD

DevOps
"ntop" conveniently displays web pages with charts and plots of your local network traffic. To record historical traffic, it uses Round Robin Database (rrd). The default ntop package enables the RRD plugin, but doesn't give itself permission to write out its data. Here's how to get the two to work:sudo apt-get install ntopsudo install -d /var/lib/ntopsudo install -o nobody -d /var/lib/ntop/interfacessudo install -o nobody -d /var/lib/ntop/rrdTo test, start "sudo ntop" (probably in "screen"), then open the web page http://localhost:3000To activate the RRD plugin, select Plugins / Round Robin / Describe, then click the red "no" to become a blue "yes." You'll see messages in the terminal where you started ntop.After a few moments, you can view RRD graphs. One is in Summary / Network Load.
Read More

Fast Parallel Downloading (for apt-get) #2

Dev, DevOps
This is how to upgrade your Ubuntu system, downloading several packages in parallel. See previous article for details:cd /var/cache/apt/archives/apt-get -y --print-uris -u dselect-upgrade >debs.listegrep -o -e "http://[^']+" debs.list | xargs -l3 -P5 wget -nvapt-get -u dselect-upgrade
Read More

Cheap Network Measurement

Dev, DevOps
You can't control what you can't measure. I want a fast local wireless network, counting only from my workstation to the router. If I change settings or drivers, how can I tell if it's an improvement or not? How can I gauge my network speed, including driver and network settings?There are jillions of tools available*, but they mostly 1) require a specific client on the server, or 2) don't actually send traffic, they passively analyze what's there.Here's a solution: transmit a largish file to the server, where it will ignore the data. Measure the network bandwidth locally using 'cpipe' or another tool.Setup: enable SSH on the router, install "cpipe" locally.cat /boot/vmlinuz-* | cpipe -vt | ssh router 'cat >/dev/null'I get lines like this, with statistics on each of many chunks…
Read More