>Test flights. My tests keep failing until I fix all of my code, then we deploy to production.

Do your tests require tens of millions, and can have potential casualties?

Running a code test doesn't require firing a rocket.

How would you test a rocket?

You test components in isolation, you test integration of components, you run simulations of the entire rocket, and finally you test the rocket launch.

You’ll catch issues along the way, but you can’t catch all of them before a full launch test. That’s why there are launch tests.

This can get as far as the test plan is complete, multiply iterated under different interface conditions and thorough. And you are still relying upon the adherence of the simulated models to the physical reality.

Real tests do all of this at once with no option to escape reality.

Again, one thing is automating thorough software tests, another one is testing physical stuff.

This is the programmer fallacy if you have a bunch of code passing unit tests, it’s going to work when combined.

Thats not what he said. Unit tests are the first stage, and are very useful at isolating the problem.

Integration tests are the next where multiple units are combined.

Then there is staging.

Did they say that?

Boeing did, with Starliner.

Test code by running it.

Test a rocket by launching it.

I would consider these launches test launches. Production is when they include commercial payloads and humans.

In production? I don't disagree that tests 'in production' are sometimes necessary (canary tests), but most of the quirks are often fixed by then.

Honestly I thought they would be live testing fuel exchange in orbit by now. Seems pretty far from it sadly.

That might still happen this year, it’s the next step in the development plan.

What makes these launches “non-production” tests is that they are not carrying any valuable payload. Blowing up rockets like this is exactly what gives the company it’s advantage over competitors who try to anticipate everything during design stages.

There was no real payload on this, so I'd argue it's closer to a QA environment than production.

It's true that other rocket companies are treating launches as production, but SpaceX has always been doing "hardware-rich" testing.

Testing their ability to deploy satellites is a short-term goal that will make them money now. Testing refuelling will be needed for Luna and Mars missions, but that’s a long way off anyway.

Some domains have so many different parties doing different things, you just have to test in production. Rockets are probably one of them.

They had that on the timeline for 2023, so it's reasonable to assume they would do it.

launching a rocket is far more analogulous to shipping a release, than it is running code.

Launching a rocket is far more complex than shipping a release.

It is more like an "all or nothing" process.

Thank god you're not building rockets.

This has been SpaceX’s methodology for a long time now and has gotten them to the point where they have the most reliable western launch vehicles ever launching record amounts of mass to orbit each year at record low prices.

Testing to failure is pretty common in rocketry. If you don’t push the limits you’ll never really know where the limits are.

I truly hope that if you ever design a rocket yourself, that you will test it. I have no idea why you'd think testing is a terrible thing to do if it has to do with rockets.

I think we all agree that you need to test eventually. I do think most of us would already be double checking for leaks. It just seems one of the obvious things that may go wrong when putting it all together.

They likely did test it, and it passed. The leak was probably caused by the somewhat violent environment of the launch, and that can’t be entirely replicated on the ground.

Why precisely? Can you elaborate?