How Much of Your Software Factory Can Run in the Dark?

A lights-out factory (sometimes called a dark factory) runs production without people on the floor. The process has been designed, the machines have their jobs, and production continues without someone supervising every step. People still maintain it and change what it does. They just don't need to be there for every run. The trade-off is that you've taken the human out of the loop, and that's not always what you want.

A lights-on factory keeps people involved as the work happens. There can be just as much machinery and automation, but people are making decisions, handling exceptions and adjusting the work. Then there's the workshop, where you're still figuring out what to make. You build something, try it, realise part of the idea was wrong and change it.

The same split applies to how we build software with agents. In our world, operational and reactive work is where lights-out can make sense. Something happens, the system evaluates it and responds within boundaries we've established. We don't want every execution waiting for a person. Product development can be different. We're deciding what should exist, trying approaches and changing our minds as we learn. Agents do the implementation, but our involvement changes what gets built. That's why we keep the lights on.

At Swamp, we run both. My colleague wrote about our lights-out operation here. Our software creation factory is lights-on, even though none of us manually write the code. I think that distinction gets missed when people talk about autonomous software factories. You can want extensive automation in both and still have good reasons to stay involved in one of them.

Operational work shouldn't always need us

There's a lot of operational work where we already know what we want to happen. An event arrives, we gather the relevant context, decide whether it needs action and take the appropriate next step. Sometimes that next step is to escalate to a person, but other times it doesn't need to.

If we understand the job well enough to define the response and check the result, I want the system to handle it. Having someone available to repeat the same investigation and make the same call every time is a dependency worth removing. It interrupts their day, it limits how much work we can handle, and it makes response time depend on who happens to be around.

That doesn't mean every operational problem is predictable. Anyone who has worked in operations knows how quickly an apparently familiar failure can turn into something else. The question is whether we've given the system enough context and authority to handle the case in front of it, and whether it knows when it has reached the end of that authority. Even in our lights-out operation, when there's a decision to be made, we tell a human. Escalation is part of the operation, not a failure of it. A lights-out system doesn't have to pretend it can solve everything.

It also doesn't mean every step has to be deterministic. AI can interpret inputs and make decisions within the process. We still need to decide what job it has, what actions it can take and how we'll know whether it's doing that job properly. There's work involved in getting to the point where we're comfortable leaving it alone.

But once we've done that work, I don't want us standing beside it. That's the benefit we were building towards. Let it run, look at how it performs and improve it when we learn something.

Building the product keeps giving us decisions

Our software creation process needs something different from us. We have a very small team building Swamp and none of us manually write code, but we aren't putting tickets into a queue and accepting whatever comes out. We discuss the approach, reject plans, use what gets built and decide whether it belongs in the product.

A ticket can look much more settled than it actually is. Take something like retrying a failed workflow. Add a retry command, write the tests, update the docs. It sounds ready to implement. But are we restarting the whole workflow or continuing from the failed step? Are we using the original inputs? What if the workflow definition has changed? What if the first attempt already created infrastructure and running it again creates a second copy?

The agent can investigate those questions and propose answers. It might spot problems we haven't thought of. It can also pick reasonable answers and carry on. Now the implementation defines what retry means, the tests confirm it and the documentation explains it. Everything agrees, but we may never have discussed whether that's the behaviour we want to offer users.

That's where I want to be involved. There are consequences beyond getting the command to work. We're creating something people will rely on, and later changes will have to account for the promise we've made. I don't want to find out what that promise is by reading the documentation after the feature has been built.

Human engineers have always filled gaps in requirements too. This isn't a problem agents invented. The speed changes how easily it can happen without anyone else noticing. A question that might previously have come up during a few days of development can become a finished feature in one session.

I described an example in The Code Was Never the Job, where an agent investigating a manifest path-resolution issue proposed an approach that conflicted with a contract elsewhere in the system. Working through that conflict changed the plan before implementation. The agent could do the investigation and development. It needed context about what we had to preserve.

I'm happy with that interaction. The plan gave us something concrete to discuss, and the discussion improved the approach. I'd rather do that than discover the assumption after it has spread into code, tests and documentation.

Some things we need to try

There's another reason I want the lights on during product work. Sometimes I don't know whether an idea is good until I can use it.

A plan can look sensible and the resulting workflow can still be awkward. You try it and realise you've introduced another concept the user has to understand, or added three steps to something that should have taken one. It might do exactly what the ticket requested. Using it gives you a better question than the one you started with.

That's the workshop part of our software factory. Agents make it much cheaper to build enough of an idea to find out whether we should continue. We can try different approaches without committing a week to each one. But we have to actually use that opportunity. If the first working version goes straight from green tests to shipped feature, we've mostly used the speed to finish our first assumption.

The agent can help with this too. It can challenge requirements, explore alternatives and bring back something we missed. I want it doing that. We still need to work through what those findings mean for the product. Does the feature belong? Is the extra complexity justified? Could we solve the user's problem by improving something we already have?

Sometimes the right response to a working feature is to change it. Sometimes it's to delete it. That's harder to do once everyone considers the work finished, which is another reason I want these conversations while we're building.

Keeping the lights on doesn't mean I'm watching files change. There are long stretches where agents should get on with implementation and I should do something else. If we've settled the behaviour and the remaining choices fit within our constraints, there's no benefit in making the agent ask me about every detail.

The checks matter in both factories

Having people involved doesn't mean the output is good. We can agree on a plan and still miss something. Both factories need verification.

This is the problem I keep coming back to with agent-written tests. An agent can misunderstand a requirement, implement that misunderstanding and write tests that confirm it. You get a comprehensive suite and a green build. At Swamp, we've had boundary checks find real defects with thousands of unit tests passing.

I covered the reasoning in Why Agent-Written Code Needs Product Verification. We need expectations that come from outside the implementation loop, and the implementing agent shouldn't be able to change those expectations to make a failure go away. Those checks can be automated. Independent verification doesn't require a person manually clicking through every release.

But go back to retrying a workflow. We could have an independently defined check proving that retry restarts the workflow successfully. That's useful if restarting is what we intended. If users need to resume from the failure, we've verified the wrong behaviour perfectly well. We have to understand that difference before we can expect the verification to catch it.

So I want both: checks that give us confidence in the output, and involvement during product development that helps us decide what the output should be. Putting a person at the end of the line doesn't replace the conversations we should have had earlier.

The boundary moves

Operational work and product work are useful starting points, but I wouldn't turn them into rules about who must be present for every task. An incident can expose a problem we've never considered. We might need to investigate it, design a response and change the system before we can handle that class of problem unattended. We've moved into development work, even though an operational event started it.

It goes the other way too. A product decision can be settled and the agent can implement it independently. I don't need to remain involved just because the ticket is labelled as a feature. I need to know when the work reaches a decision that needs more context or discussion.

And if I keep answering the same question, we should fix that. Write down the constraint, improve the agent's context, add verification where it helps. Repeatedly supplying information the system should already have is a poor use of anyone's time. The software factory should get better at handling what we've already learned.

What I don't want is to treat lights-out as a maturity score for everything we do. Some interactions are waste: unnecessary approvals, repeated explanations, waiting for somebody to run a command the system could run itself. Others are where we discover that the plan is wrong or the product needs to change. Removing both would make the process quieter. I'm not convinced it would make the product better.

At Swamp, our lights-on software factory builds systems we want to operate with the lights off. We work through the responsibility we're giving those systems, build it into the operation and let them run. When something new comes back, we have more work to do in the software factory.

That arrangement gives us time back. I want us spending it with users, trying the product and thinking about what we should build next. We could use agents to get through more tickets instead. I'd rather use them to give us more time to decide which tickets should exist.