Sorry, I can't be bothered trying to write something technical in French right now. But anyway:
The mid vowel pairs /e~ɛ, ø~œ, o~ɔ/ are mostly in complementary distribution in French, yes. Mostly they are close-mid when in open syllables and open-mid when in closed syllables. There are exceptions though so they are all phonemes, though with a small functional load I guess. There aren't a lot of minimal pairs, especially for /ø~œ/. They also tend to not contrast at the end of a word where only /o/ and /ø/ are found. (Note /e/ and /ɛ/ do contrast here though, like in some verb inflections.)
As a rule, <eau> and <ô> are always /o/. (In a few words with <ô> it may vary though.) <au> and <o> tend to vary more and are less predictable. There are exceptions to the loose rule about closed vs. open syllables: the vowel tends to be /o/ and /ø/ before /z/, for some reason; and /ɔ/ tends to be found before /ʁ/. (Wikipedia also says it's /ɔ/ before /ɲ/ and /ɡ/, and I guess it is, like
oignon). If the vowel is followed by two consonants in spelling it will very likely be open-mid whether they are still pronounced or not, such as double consonants that have become degeminated (like
lotte). If there is a single consonant followed by a mute <e>, treat that as an open syllable and raise the vowel, like in
haute (with /o/).
If you follow these guidelines and use your ears you should be fine, since again these vowels only lightly contrast anyway and mostly only in monosyllabic words. There is also some dialectal variation (many words with /ɛ/ in France have /e/ in Quebec French) and some words have free variation too. I think French speakers would be used to this, so it's not the end of the world if you use the "wrong" vowel here and there. (Trust me, Quebec French does much worse things to some vowels.)
For an analogy, English has some words where /ɔɹ/ is pronounced by some speakers as /oʊɹ/ and words where /ɛɹ/ may be pronounced /eɪɹ/, and it doesn't matter because those vowels don't really contrast in that position. It might sound like you have an accent, but it shouldn't make you that hard to understand as long as you don't just swap vowels around at random. But they are mostly predictable, and you can memorize the exceptions, so it should be easy.