Gregory's Server@nuthatch @danluu They’re not all terrible at software. A car is not a phone on wheels. A car is not a Windows PC on wheels. A car is not a web server on wheels. The problem domain is a lot more difficult than anything you’ve encountered in mainstream computing. For a start, if it goes wrong then people can be injured or die.
|
62 comments
 @kentindell @nuthatch @danluu Well, engineers are there to solve the problem based on its environment, not to let cars fail on software upload because it, maybe, rained. What you are REALLY saying is that the automotive software industry is choosing tradeoffs that will leave some people stranded. @kentindell @nuthatch @danluu Are you saying there are no tradeoffs in engineering? Are you saying there are no tradeoffs in automotive engineering? Are you saying this screen, displaying an error message designed to fit the situation, is somehow not subject to tradeoffs in engineering?  @kentindell @ashmueli @nuthatch @danluu I would hope a problem with actual safety while moving is the root of this. Being stranded could be a safety issue. Vehicles have been computerized for decades and have always had a limp home backup for this reason. I would regard this as a design problem. My career covered analog to bus systems. @kneworldodor @kentindell @nuthatch @danluu Exactly. It’s a design problem and, unlike, say, space flight, car designers have the option to allow for grounding the car. The question now is, what are the tradeoffs behind this particular case. @ashmueli @kentindell @nuthatch @danluu the specifics are important and I might agree with a shutdown if I knew them. I see no reason that most systems can't shut down or resort to an open loop program 99% of the time though. If it's related to protecting from electrical fields during reflash or network access so it doesn't get bricked during update then it's a stupid mistake. @kentindell @ashmueli @nuthatch @danluu nd of course this could easily be a hardware fail and AFAIK cars don;t use secondary redundant hardware and evne if they do and a primary hardware fails, you'd still ground the car when stationary @peterainbow @ashmueli @nuthatch @danluu There is often redundancy: ECC memory, lock-step CPU cores, multiple bus paths. The silicon has fault injection for built-in tests at start to check this. @kentindell @ashmueli @nuthatch @danluu that's not the case on the older gneration of car computers, but i'm guessing things have moved on, any pointers to where i can learn more ( jst for learning sake ), finding the search engine world pretty much borked again these days lol @peterainbow @ashmueli @nuthatch @danluu Yes, search is grey goo now. But I can point you at a blog post about updating firmware in cars. https://kentindell.github.io/2023/04/18/get-your-app-to-mars/ @kentindell @ashmueli @nuthatch @danluu oh i've done some of that from pi's and webcams to printers and even connecting to FIAT ECUs, but 2000ish models did not have ECU redundancy at all, just wondering if that's changed. obviously they probabky use the backup swtchover flash system, but if something fails (hardware) then that's it and as a firmware dev there are diminishing returns in showing the end user low level error info, better to go for the don;t panic general error screen as seen above @kentindell @nuthatch @danluu People get killed by software all the time. See https://archive.org/details/lca2019-Im_sorry_Dave_I_cant_do_that_Ethics_in_Software_Development @kentindell @nuthatch @danluu Thank you for totally missing my point, completely not looking at the resource I provided in support of my point, and just restating your point. @Ooze @kentindell @nuthatch @danluu Immediately followed by the drive-by block... Don't have to worry about seeing his particular mix of hostility and self centeredness again  @kentindell @Ooze @nuthatch @danluu People still get killed by safety critical software all the time, see that guy in South Korea who was killed by a packing robot a few weeks ago because it thought he was a box. “Safety critical software” doesn’t mean it’s safe or it works 100% of the time, it means that when it fails people can die. And it does, and they do. @kentindell @nuthatch @danluu well i guess a car that wont move is probably safe...but its hardly a graceful failure if the fall back is just to break everything Its also a piss-poor user experience @OliverNoble @nuthatch @danluu That’s how it’s supposed to work. Safety comes first, then comes reliability.  @kentindell @OliverNoble @nuthatch @danluu Part of safety is not pushing an update that breaks the system in the first place. Plus when an update is performed >>NOT PUSHED<<, the system verifies that it received the update correctly, checks signatures, then runs the software. If there is an error, it reverts to previous state >>WITHOUT USER INTERVENTION<< A vehicle which fails to move because the Mfg pushed software is unsafe, not unreliable. @kentindell @OliverNoble @nuthatch @danluu Not really. A quick dive into forums shows that Ford has a fully embuggerred software update system, either failing to install critical updates, or bricking vehicles. There seems to be a trend of incompetent/nonexistent dealer support and a lack of concern at the Mfg level. I hadn't really thought about buying a Ford, and I've been hesitant to buy anything with push OTA update. Ten minutes of reading forums convinced me to avoid both. @johntimaeus @OliverNoble @nuthatch @danluu If you think it’s easy then you don’t understand the problem. Like why cars need to be parked on level ground to be updated. Or why it entails new microcontroller silicon designs. @johntimaeus @OliverNoble @nuthatch @danluu It’s not bricked: it needs to be connected to an authorized diagnostic tool. But clearly you don’t want to learn and I don’t want to hear uninformed opinion, so on that basis I’m muting this. @kentindell @johntimaeus @OliverNoble @nuthatch @danluu  @johntimaeus @kentindell @OliverNoble @nuthatch @danluu it's possible that they are required to brick the vehicle in the case of a failed update. Anything else could involve the vehicle running in an indeterminate state or with a known issue. Both could be liability issues Of course ideally the update shouldn't fail but that's impossible to guarantee. @Wearwolf @johntimaeus @kentindell @OliverNoble @nuthatch @danluu  @johntimaeus Yeah... I don't want OTA push, I don't want internet connectivity, I don't want ANY data sent to the manufacturer without explicit consent, I don't want a car that requires security updates, I don't want tons of sensors and helpers (except perhaps a camera in the back), I don't want "smart" or keyless, I don't want touchscreens everywhere. But I do want an EV. Will there be something like this in the next decade (or 25 years)? 😔 Time for a better bike. @jesterchen @johntimaeus There is a dedicated fuse in all these cars for whatever connects them to the internet and you can look up in the owner's manual which fuse it is. Then you can take it out. I am pretty happy with our Honda Clarity. PHEV, 35ish mile range on pure electric. The gas engine is a descendent of the standard Honda 1.4L straight 4. Elegant drive model: 1 reduction gear, 1 electronically controlled clutch for main power, 2 clutches in the differential. The infotainment/display system is isolated from the operational bits. Software update requires a physical cable. @johntimaeus @OliverNoble @nuthatch @danluu @kentindell and if you can’t do this automatically, then schedule the updates when the car is in a workshop, with a technician who can resolve it before handing the car back. @OliverNoble @kentindell @nuthatch @danluu it depends on what the car was doing when it stopped being drivable. Its been shown multiple times that you can maliciously alter the software of a car that’s actively driving (multiple manufacturers), which means a lack of protections against changes at “bad” times @masukomi @OliverNoble @nuthatch @danluu There are security threats to cars while driving eh? 🤔 I think you could be right.  @kentindell @nuthatch @danluu "if it goes wrong then people can be injured or die" Only if you try hard to make it that bad. The aircraft I've flown, all the computers can fail, all the screens can go blank, and I'd still be able to land the thing safely. There's no obvious reason why cars should be any worse. @TimWardCam @nuthatch @danluu And I bet if your pre-flight checks fail you don’t take off. @kentindell @nuthatch @danluu Yup. I've rejected an aircraft more than once. Once, for example, because I spotted a tiny dent in the tailplane that nobody else had seen. Presumably it had been bashed by something in the hangar. Had this been a hard enough bash to break or weaken something structural inside? - I didn't wish to find out, that's what engineers are for, not customers. The next time I rented that aircraft the dent was no longer there.  @kentindell @nuthatch @danluu there is so much wrong with these statements. 3. If problem domain so more difficult, then making updates not break system should be a breeze. Even if update is broken for some reason, entire High Avaliability world knows to boot from backup for more than 20 years. Great, did it brick engine controller? So, do you still belive such incompetence will not result in injuries or deaths? If someone cannot write software update in such way that doesn't turn dangerous hunk of metal into useless dangerous hunk of metal, said one should not be allowed to write critical software for it. @kentindell @nuthatch @danluu Do you care to elaborate? I'm not discounting the complexity, but I'm definitely curious about how something would be so catastrophically wrong the car couldn't flash back (or at the very least, allow the car to function in a reduced functionality state) @ollien @nuthatch @danluu It’s possible one of dozens and dozens of ECUs couldn’t roll back (assuming they all have the flash to roll back). They all have to be set consistently, and it’s possible that one of them failed to take a security key update to its HSM. It might be a corrupted memory problem on an ECU (HSMs that lose power during a key update get very finicky because they’re trying to protect against a glitching attack to force a key rollback). @kentindell @nuthatch @danluu Makes sense - this is definitely far removed from my area of expertise and I appreciate the insight :) Would it be too costly to have redundant systems of some kind to prevent this sort of failure?  @kentindell @nuthatch @danluu Dude. It's an updater. It should be *entirely* orthogonal to the actual function of the code being updated, unless you're daft enough to think the updater should be able to run when the vehicle is in motion. BTW mission critical updating is a problem that has been solved, the actual point here is that the manufacturer didn't implement that, which calls the operation of their actually mission critical competence into serious question. @kentindell @nuthatch @danluu Oh by all means do be specific. I've only got a few decades experience in telecom (which targets a MTBF of 400 years), so I'm interested in learning exactly what I've missed here. @kentindell @nuthatch @danluu Or how about you dodge the details in an attempt to cover. 😂 @kentindell @nuthatch @danluu No, you're supposed to offer a vaguely cogent argument as to why a car manufacturer can't implement a robust update system while a vehicle isn't engaging in real time mechatronic functions (other than someone assessed that the probability of an update failing and the resulting compensation/PR grief was less than the cost of adding a second boot PROM to the board). Or in basic terms, you made a point; offer some minimal modicum of proof. |
@kentindell @nuthatch @danluu But this is not an alien concept, right? Not all computing is end-user tablets. We use computing to fly to space, run trains, medical equipment. Do we see screens like this a lot in the operating room?