What product funding doesn’t mean

As we seek to move our funding system to a fundamentally different model, here are two misapplications to avoid.

Well, my last post on product vs. project funding was well received. Thank you to everyone who sent it along to others, shared it, and reached out with stories about how you are using the deck. Fantastic stuff going on out there!

There’ve been two important caveats that I now realize should have come with the post. I’ve said them both before but they bear repeating. The first is very simple. If the product you need already exists, for the love of God, please don’t build it. Just buy it. Find a way to buy it even if there are ways in which it doesn’t do exactly what you want. I wrote about this in my book: 

Government can, and should, buy commodity software products from companies to handle its basic internal needs, which are not all that different from the needs of the private sector: communication tools, HR and payroll systems, and so on. Indeed, government procurement teams have a terrible habit of contracting for bespoke software when they could buy commercial products—partly because we tell these teams to collect every possible requirement they can think of, which encourages and even calcifies arcane practices within the departments they serve. Those practices might feel like they have the force of law, but a closer look will often reveal them as mere clutter. An accounting department within the Department of Defense, for instance, requires custom software to follow intricate DoD bookkeeping guidance. The need would be much better served by having the DoD simplify its bookkeeping practices to work with standard accounting software instead.

That last bit of advice is ensconced in the Software Acquisition and Practices (SWAP) Study the Defense Innovation Board wrote for the Department of Defense: 

For commodity functions, “Unmodified commercial software should be deployed in nearly all circumstances. Where DoD processes are not amenable to this approach, the Department should modify its processes, not the software.” 

In other words, enough with bespoke requirements for commodity functions. Stop telling your people to do a better, more thorough job of the wrong thing. But how do you know what to build vs. buy? A good framework for this is Wardley mapping, which I wrote about seven years ago (!) under the provocative title The Tyranny of Agile, referring to the problem of applying agile development to something that doesn’t need to be developed at all. Either way, what you want is a product, something that’s being constantly invested in and updated. Something that won’t rot. You’re just trying to decide if it’s you or someone else who’s in charge of that constant investment. If the product is core to your agency’s mission, the technology needs to be your product. You need to own the code, and you need to be able to change it to meet changing needs. If it's something everyone else does, let that investment be someone else’s problem.

The second caveat is that I didn’t describe a very common (and healthy) pattern with the early stages of products, which is that they fail. The first discovery sprint does not necessarily lead to a clear understanding of the need, or sufficient alignment around that understanding. What you do then is stop and assess the situation. It may be that there isn’t really a need, or that the need is best met through other means. It may be that the need is clear but the product vision is not. It may be that this particular discovery sprint team wasn’t right for this particular job, and it needs to be restaffed with a different group. It may just be that conditions are not right to move forward for any number of reasons. So you stop. And likely then go again later. But not always.

Code for America’s tax portfolio had three discovery sprints before we went forward with product development (it may have been more…it’s been a minute). Back around 2018 or so (I left the org in early 2020), our team was supposed to have been finding benefits in the workforce development arena to which we could apply the model of our breakout success product, GetCalFresh. But we’d also set a goal of doing something that would actually help people, not just digitizing benefits with minimal impact. Turns out there’s not a lot that’s really meaningful in that world. Workforce development agencies aren’t very effective at helping people get jobs, and we just couldn’t seem to figure out a way to make them so. These agencies often administer benefits like money for bus fare to job interviews. That’s just not going to make a meaningful difference. 

But after a few tries of teams coming back with no really compelling opportunities, a third small team (I think it was 2 full time people and some extra help for 4 or 6 weeks) landed on a key insight: people looking for jobs benefited enormously when they got a check from the IRS because they qualified for the Earned Income Tax Credit. It was often a couple of thousand dollars, and when people used it to, say, get their car fixed, that was often the thing that enabled them to get a new job. Lots of people were eligible for the EITC but not getting it, so getting more low income people to file taxes was a real opportunity. EITC isn’t technically workforce support, so this team was straying outside of the assignment, but the opportunity was true to our North Star and a great fit for our organization’s competencies and capacities. So we finally moved out of discovery and assigned a slightly larger team to test some assumptions. It still wasn’t a full development team, but at that point we were officially past the first gate.

That led us to start working with VITA (the IRS’s Volunteer Income Tax Assistance) to help people claim their EITC, which put us in the position to help VITA’s transition from in-person to digital during the pandemic, which led to the opportunity to provide a way for non-tax-filers to claim the Child Tax Credit during the pandemic, which led to being the state implementation partner on the roll out of Direct File (all the last bits happening after my tenure there.) 

In retrospect, this initiative has led to a variety of highly successful products, and if you know the end of the story, it can seem obvious that stopping, resetting, and starting over again – three times! – was the smart thing to do. And it was easier to do in our non-profit than it can be in government or other settings, because of our culture and stated practices – this was literally what we had designed the org to do. But even under these conditions, it can be hard. I give the staff credit for great instincts – they knew when it wasn’t right, and they also knew when we finally had something – but it’s impossible not to worry about telling a funder, for instance, that the thing they funded wasn’t going to become something amazing (at least not yet.) You have to be nimble at moving staff around when you do this many short sprints, and concerns about staff allocations can cause angst. In the moment, giving something a thumbs down can feel stressful even when you know it's the right thing to do. 

The story I told with those graphs in my last post is not just fictionalized (as I warned) but also idealized. Sometimes the insights from a discovery sprint give you the confidence to invest in a minimum viable product, and sometimes they don’t. Sometimes your minimum viable product shows enough promise to merit further investment, and sometimes it doesn’t. The problem with the project model is that you’re not testing your assumptions along the way. (Cue Clay Shirky’s classic line “The waterfall method amounts to a pledge by all parties not to learn anything while doing the actual work.”) If you took my charts literally, you would assume that each stage leads seamlessly and inevitably into the next. That the next stage is NOT inevitable is the entire point. Sometimes the red line on the graph isn’t a relatively steady up and slowly to the right, but looks like this, as you try different discovery approaches:

And yes, that red product line is both more chaotic and more expensive than the blue project line in that time frame, but zooming out, of course, it’s going to look a lot better, and get better results.

Here, by the way, is another way of envisioning those stages, as we used to articulate them at Code for America. (This is about seven years old, and I’m no longer involved with the organization, so I am not trying to represent the organization’s current strategy or operations. Also, these are not the exact same gates or phases a government tech product might go through because though we were working on government transformation, we were doing so from outside through partnerships.)

The goal is to move from left to right through these stages, but we do ourselves a disservice when we make it look so linear. Getting “stuck” in one of these earlier stages is totally normal and healthy. And cheap! Certainly a lot cheaper than funding full development teams for projects that are doomed to fail. 

Put another way, the point of this system is to avoid building the wrong thing, and to invest the right amounts at the right times so that you build the right thing. This requires a variety of actors in the system to share the same assumptions. If both the people asking for the money and those approving it see stopping and starting as signs of failure instead of signs of learning and de-risking, there will be strong incentives to keep to the idealized trajectory, even when a messier trajectory is needed. We can blame those in charge of the purse strings for not getting it, but those asking for tech investment need to hold themselves accountable to these principles, too. Don’t ask for the next level of investment just because you finished a discovery sprint, or a minimum viable product, or whatever. Ask for it only when you know that it’s time to move to the next level. Train the people with the purse strings in a new model, but also train them to trust you. 

***

I started this substack writing about I-95ness, but have recently turned to government technology topics, and a lot of you have subscribed in recent weeks. Warning: I’m now going to return to state capacity topics. I see government’s digital competency as a part of this larger issue. We’re also going to talk about the relationship of our crisis of state capacity to this political moment. As they say, stay with me.