extra Docker networking goodies

Uncategorized
I work a great deal with Docker, connecting webservers and databases together, hiding everything behind a caching proxy like Nginx.  Docker is a wonderful tool, providing isolation and semi-magical capabilities like hiding an entire "cluster" of machines behind a single container, a single process. However containers are not much fun. They're generally so stripped down as to make debugging issues rather more awkward than normal.  The following questions are common when something doesn't work: is my process running on the correct port? what processes are running? which services are up, on which ports? With a default Docker container, answers to these questions aren't obvious. Solution: add the following to all of your Dockerfiles: RUN apt-get update && apt-get install -y apt-file curl net-tools procps psmisc # 'psmisc' provides /bin/fuser; procps=>/bin/ps;…
Read More

modest makefile for rapid Java testing

Uncategorized
This makefile is designed to help test several Java programs, each one in its own self-contained source file, for example "hello.java" If you type "make hello.test", the program will be compiled and executed. To test everything in the directory, type "make test-all". To rebuild everything then re-test, do "make clean test-all". To test a new program, just start writing it. When "make test-fast" is executed, the class file will be built, then run. That is, you never have to update the Makefile -- it picks up the most recent Java source file automatically. I bind "make test-fast" to the big keypad Enter key, but that's for another blog post. RECENT_JAVA := $(shell ls -1t *.java | head -1) all test-fast: $(basename $(RECENT_JAVA)).test test-all: $(patsubst %.java,%.test,$(wildcard *.java)) %.class: %.java javac $<…
Read More

TIP: language-aware Git gives you contextual searches

Uncategorized
TIP: if Git is setup correctly, Git knows we're working with Python, so commands are clearer. setup: echo '*.py filter=python' >> .gitattributes Example: find the usage/definition of formatted_tax: $ git grep -p formatted_tax app/models.py=2083=class Order(CreatedMixin): app/models.py:2454:            'tax': self.formatted_tax, The second line says the symbol was found on line 2454.  The first line shows the context, the symbol was found inside the class Order! Very useful related: 'git grep' and Language-Aware Diffs
Read More

TIP: running 2N+1 Gunicorn workers

Uncategorized
If you read, people tend to say that for web services, run 2N+1 workers, where N is the number of CPUs. It turns out the Intarwebs are wrong about exactly how to do this. Here's my solution based on "epicbrew"'s work on Stack Overflow: gunicorn --workers=$((2 * $(getconf _NPROCESSORS_ONLN) + 1)) wsgi:application The cool thing is that it works on both macOS and Linux! Ref: What is the appropriate number of Gunicorn workers?
Read More

Ego is the enemy of great work

Uncategorized
Today coincidentally I listened to two strong podcasts that highlight the enemy of great work. Not fear, but EGO. In the first talk Andy Molinsky talks persuasively about how to move forward in your career when everything is overwhelming and/or you're an introvert. Example: you have a small business and you feel uncomfortable 1) meeting new people, and 2) promoting yourself. Of course, with a small business, a huge way to get your name out there and to get new clients is to go to "business mixers", where talking to strangers about how great you are is all you do.  His solution is simple and direct: define "success" before you go into a challenge, and repeat to cement incremental progress into large gains over time. Example: define "success" as making…
Read More

code: sync files using FTP

Uncategorized
My webhost doesn't support rsync, only FTP.  For grins I wrote a script to scan through interesting local files and push them up to my webhost.  Alas the progress wasn't as trivial as I'd hoped, so I abandoned the effort and used lftp the "mirror" command. The code shows a number of best practices: don't store passwords/keys, pass them in via environment variables make an iterator of files, add to it using chain() Python3 subprocess.check_output() is a little odd logging, like voting, should be done early and often code ensures FTP connection always closed cleanly https://github.com/shavenwarthog/johntellsall/blob/master/class/django-queryset/ftpsync.py
Read More

talk: Functional Programming and Django QuerySets soon!

Uncategorized
I'll be speaking at the next SoCal Python meetup! http://www.meetup.com/socalpython/events/230329648/ If you want to go, sign up soon, there are only a few spots left. This talk is mostly about Functional Programming in the Python world.  FP is great for testing, but it can be a bit mysterious at times. I highlight the awesomeness of FP, the dangerous bit, and how it is related to Django QuerySets. I've given this talk a few times, and people always get excited about it. FP is not magic!  It's a clean and graceful way to write code. Here's a link with my previous version's slides, notes, and other references: http://johntellsall.blogspot.com/2014/05/talk-functional-programming-and-django.html Each time I give a talk I change it up a little or a lot. This time I'll probably add more cat photos.
Read More

TIP: auto-fix your Python code!

Uncategorized
The script autopep8 will automatically fix and reindent your Python code! The following shows a "diff" of what PEP8 changes should be made, and then we apply the changes: $ autopep8 -d collect.py @@ -29,5 +29,5 @@ -if __name__=='__main__': +if __name__ == '__main__':      main() $ autopep8 -i collect.py
Read More