You're absolutely correct.
All old weak masculine nouns get that treatment in compounds, they usually lose that final -e and only the stem is used. Cf. kudde - kuddvar. Those nouns end in -e in singular, -ar in plural, and are common gender today.
The reason it also loses an m has to do with spelling, for some reason we often drop the second one in cases like these where a morpheme that starts in a consonant is added right after it. It makes no difference whatsoever for the pronunciation, so my best guess is that someone thought two m's looked clumsy and/or took up too much space, and since you could do away with one, you did.
read fluently, understand well, speak badly;
read fluently, understand badly, can't speak;
read some, understand a bit, speak a few sentences;
heritage language, want to understand and speak but can't.