a basic strategy (write-ahead-logging) is
- write to disk that you're going to do thing A with parameters B -- ensure this itself is fault tolerant eg with sqlite
- once the log is committed, you can safely return to the caller. don't return before then
- do the thing A with B
- report results
- remove the log entry
finally, on program startup, go through the current log and do everything in it
@haskal in practice oftentimes one can skip the last step of removing the log entry and use a stack of a certain upper bound for the log, then go through and ensure they've all been committed the next time it is mounted.