@mhoye I have been using project-oriented history for a while too, though it's based on named projects instead of directories.
I tend to leave a terminal tab open for most long-term projects, and the history module prods me for a project name when I open a new one. It also restores the right one for a given terminal ID on restore.
If it's an existing project, it'll switch to the directory I was last using for that project. It also has a little per-project init file.