But "us" is not formed by conjugating "we" in a regular way - it is an entirely different word.
And this goes all the way back to the Proto-Indo-European language, thousands of years ago: The subject form was *wey and the object form *nsme (sort of, possibly, probably).
English, German, Norwegian etc. have inherited both forms, with more or less the same function.
But in the red languages, the object form took over the function of the subject form. Like latin "nos" and its descendants.
Another thing that has happened in several languages, is that w has become m - like in Western Norwegian. The reason is that verb endings have influenced the pronoun - Old Norse "erum vit" ("we are"/"are we") became "eru m(v)it". I expect something similar happened in Slavic and Yiddish.
Some languages, like Sami and Slovenian, have a separate word for "we two" - this map does not show these words.
Also, some languages have separate words for "we (including you)" and "we (but not you)"...