Gregory's Server
I'm saying it should be open source – and reviewable by third parties – as of the time it is first used in human trials. We need to start treating software that affects human health and lives the same way we treat drugs and medical devices.
Software involved in any high risk situation should be:
1. Open source and reviewable
2. Certified by either the government or an NGO funded by insurance companies (think 'UL' certification)
We do these things with drugs and hardware now.