Package Managers

Dev, DevOps, Security, Testing
I’ve wondered why Npm seems to have so many security fires compared to e.g. the Python package system. They shouldn’t be that different. Anyway, there are reasons: https://blog.acolyer.org/2019/09/30/small-world-with-high-risks/ I'm amused that new Pip is basically adopting the lockfile from Npm, for exactly the same reason: nailing down transitive dependency versions. If package A wants package B, and B doesn't specify exactly what its dependencies are, then installing package A at various times will have unpredictable results. Example As a developer you use package A, and specify its exact version. You create lots of awesome code and tests, and everything is wonderful. Then you send your code to CI... The CI system of course rebuilds everything from scratch. It installs package A, and dependent package B. Package B has been updated since the time you…
Read More
Testing Pyramid vs… Testing Trophy?

Testing Pyramid vs… Testing Trophy?

Testing
I was honored to chat to a bunch of professional QA people last night about Testing Pyramid and Consequences, details on a previous blog post. People last night seemed interested and curious in the "Testing Trophy" concept. That is, in contrast to the Testing Pyramid, tests should be mostly integration, with a few UI and unit tests. In this way we get the most business value for each test we write. I'm a fan of the Testing Pyramid (lots of unit tests), however I'm an even bigger fan of paying really close attention to business value. Tests aren't free, and they aren't cheap! They can have bugs, and can be over-designed, so they function as "change detector" as opposed to a safety net. From Kent C Dodds: "Testing Trophy A…
Read More
new talk: Testing Pyramid and Consequences

new talk: Testing Pyramid and Consequences

Testing
I'm giving a new talk!  This Wednesday in Pasadena for the LA Software Testing meetup, and again soon in Santa Monica at the Testable LA meetup.The traditional testing pyramid is a useful tool for investing in tests which deliver business value... but there are a lot of subtleties. In this talk I'll highlight each area of the Pyramid, then discuss extensions and variations, so that we all can more fully deliver wonderful quality code, quickly!Slides on google docs: Testing Pyramid and ConsequencesHere's a recording from the LA Software Testing meetup.Resources:Meetups: Testable LA, LA Software Testing Brian Okken - Test and Code podcastMartin Fowler - Practical Test Pyramid articleBooks: Refactoring, NoSQL DistilledLisa Crispin: Agile Testing booksSafari Online books, training videos
Read More

Quality Code in Practice

Testing
Developers like to write code. They like to type fast, to work fast, and deploy fast. This is fine. We all like this, it's fun!A more high-level view is that as a business, as a team we want to deliver value fast. This is harder: which value to what audience are we serving? If we deliver buggy code to the client, no one is happy. If we make some users happy but our core audience is not served, then we haven't really delivered on our goals. Our goal as professionals, as entrepreneurs, is to find/create an audience, and serve that audience by giving them value.In practice, "clean" code helps us work fast and also deliver value to our audience. Clean in this case meaning free of bugs, clearly designed, and…
Read More

Testing Pyramid and Feedback Loops (Platonic Solids of Quality)

Testing
This talk was given 6/14/2018 in sunny downtown Santa Monica, at Carbon Five.  Rit Li's Testable group is all about, well, testing, especially for webapps. Given my focus on being an expert at "quality, dev, and devops", this group is perfect for me!Here's the event link => https://www.meetup.com/testable/events/jsjqjpyxjbkb/Here are the slides, in Google Docs => Testing Pyramid and Feedback LoopsThis page is http://bit.ly/jta-platonic2 ,  I'll add more to it as different versions of the talk appear. Please send me photos!
Read More

quality interactive tests with Lynx, the server-side browser

Testing
Working on a web page test on a server, but can't view the page in a normal web browser? It's easy enough to transfer the HTML to your local computer, or... use a text-mode browser!The Lynx browser is tiny, fast, and supports a good part of modern web pages. It's a great tool for quickly getting a rough idea of what your page looks like.Example: my Flask app refuses to handle authentication correctly. So, I write a API test to log in to myserver, capture the response, then write it to a server-side HTML file.  Next I use Lynx to view it so I can see what it's doing!Here's my Registration test:def test_post(client):    "validate page that uses POST"    rv = client.post('/auth/register', data=dict(        email='test@example.com',     …
Read More

practical use of GNU Makefile for testing

Dev, Testing
Recently I reviewed a bunch of code from students. The projects are in Java. To test each one, we:compile Java source file into class filerun the class fileexamine outputOver time I'll be getting more code, thus I want a simple workflow that will automatically adapt to having more source in the same directory. I can't just list all the source files then write out a Bash script to compile + run everything, for example. As new code comes in, I want it to be included in the overall test run.How would you accomplish this?GNU Make is the bombI used my good old friend, GNU Make! This simple tool runs programs to turn files into other files. It understands dependencies, and won't do work unless it needs to. For example, it'll compile…
Read More

TIP: use Strace to debug issues inside Docker

Dev, Docker, Testing
Yesterday my Docker application wasn't working correctly -- the appserver is hanging.  Debugging this is a challenge: there's no crash nor stack trace to point out the issue.  Is the appserver misconfigured, so it's trying to talk to a non-existent database? Is the config okay, but the network is not set up correctly?  Can an external service not see our Docker container correctly?To debug this I used my good old "strace" command to trace exactly what is happening. It outputs log messages for all system calls the appserver does, including all the network I/O.  Alas it didn't work for me:strace: test_ptrace_setoptions_for_all: PTRACE_TRACEME doesn't work: Operation not permittedThis is odd, as the Docker container is running with root permissions, and the parent container is Debian.My buddy Loren says this is a…
Read More