So I've never actually managed to read a whole book in Chinese thus far, which means I can't really say from personal experience what it's like to be able to do that. However, I will say that the difficulty I'm talking about is not necessarily with the writing system per se but rather with actually reading entire books in the language, which to me is a completely different matter.
Most of my progress with reading in Malayalam is stuff I've actually documented here on UniLang, so I can say more about that. Typically, the cultural background of the characters is different enough from my family to throw me off and really bog me down. My family is traditionally Syrian Christian from southern Kerala, but most families in Malayalam literature seem to be from central or northern Kerala and typically Hindu (sometimes Muslim, though I haven't read full novels with Muslim characters yet).
The first novel I read in Malayalam, a few years before I joined UniLang, was actually a kid's story. I read it out loud to my dad (because me reading to him is something we both enjoy) every morning/early afternoon while eating at the kitchen table before going to the university. It was pretty simple stuff, just about the everyday life of a more or less ordinary Malayalee child when my parents were growing up, and it had lots of amusing passages. Even so, though, IIRC it took me almost a year to finish reading it. It was likely set in northern Kerala, so there were words that were thrown around a lot that I was completely unfamiliar with. For example, one word that occurred a lot was കൊണ്ടാട്ടം [kɔɳˈɖaːʈəm], which apparently can refer to various types of small, deep-fried, savory snacks but isn't a word we use in our community and thus isn't a term I'd ever heard otherwise. Sometimes, there were also cultural aspects that my dad had to explain to me; for example, another word that showed up was കുഞ്ഞിപ്പലക [kuˈɲipələga], literally something like 'baby plank', because a carpenter comes to the kid's house and he asks the carpenter for one. When I read out a passage that contained that word for the first time, my dad said something like "oh, I get what they mean by that." He explained to me that traditionally (in fact, until very recently), Malayalees didn't have tables, chairs, or dining rooms and instead ate on the floor of the front porch. Food was, of course, served on a banana leaf, but if you didn't want the leaf directly touching the floor, you could have a small plank of wood with legs so small they were barely off the floor (shorter than an ordinary stool) underneath the leaf. This kind of plank was easy to make with leftover wood.
After I joined UniLang and was ready to tackle my second novel, I started to try reading Chemmeen; I already knew the plot because it was made into perhaps the most famous movie in Malayalee history, which I'd already seen, plus the movie was faithful to the novel. However, soon after I started, my dad instead suggested I read another, much shorter novel called Yakshi, which I polished off in about a week. He had already sort of spoiled the plot for me years ago, and the plot is pretty simple yet still interesting; it's basically about a ladies' man whose face becomes horribly disfigured, so when a woman becomes his girlfriend, he becomes increasingly convinced that she's actually a demoness who he has to kill before she cannibalizes him.
Then I went back to Chemmeen. Although it's set in a fishing community very close to my parents' hometown, the dialogue is all in eye dialect, and the representation of the fishing community's dialect in the novel differs wildly from any other variety of Malayalam I have ever seen. Syllables that I would normally expect to be unstressed or at least short are represented in the dialogue with the symbols for long vowels, and sometimes, the ones I would normally expect to be stressed or long are represented with symbols for short vowels. For instance, I just opened it up to a random page and picked a quote I could find there, and the way I would pronounce it would be:
വല്ല അപകടവും പറ്റിയോ, മോളേ? [ˈʋəlla əˈbəgəɖəʋʊm pəttiˈjoː], [moːˈɭeː]? 'Daughter, did some kind of danger/accident happen?'
മോളേ [moːˈɭeː] 'daughter!' is short for (i.e. a contraction of) മകളേ [məgəˈɭeː], and അപകടവും [əˈbəgəɖəʋʊm] could easily be contracted to അപകടോം [əbəgəˈɖoːm]. However, in the novel, the quote is represented in eye dialect as follows:
വല്ല അപാകടോം പറ്റീയോ മൊകാളെ?
which would suggest the pronunciation "[ˈʋəlla əbaːgəˈɖoːm pəttiːˈjoː mɔˈgaːɭe]?"
The eye dialect in this novel threw me off so much I wearily asked my dad whether there were any novels we had that were set closer to home (to our own community). He pointed me to another novel called Malagal set in our community and written by a family friend. It's about a theme that appeals a lot to him, namely entrepreneurship (leading to urbanization) in the form of a rubber plantation and joint-stock company in the previously untapped hills to the east of where we live, but it was a lot more boring for me since it offered much less in terms of interesting cultural differences or cultural information. I read both of these in tandem (i.e. one chapter of Chemmeen, then one of Malagal), hoping that the flaws of each might balance each other out. Instead, it took me a year and a half to finish reading them.
The next novel I read (this time out loud to my dad) was Pithaamahan, which was long and had lots of big words but was also very lighthearted (one of the minor characters is actually none other than Richard Nixon, in the role of an ordinary local farmer post-Watergate) and constantly cracked my dad up. It didn't have any unfamiliar varieties of Malayalam, although it did have one (somewhat minor) character who spoke Tamil. I'm pretty sure I didn't read it all in one stretch, and I even started typing up my grandfather's war memoirs by hand somewhere in the middle, but I still managed to finish it in less than a month. Needless to say, typing, reorganizing, and translating the war memoirs took a few years, especially because I also started memorizing an epic poem (which I continue to struggle with to this day and never even got halfway through with).
The last one I read was Randidangazhi, which is by the same author as Chemmeen and again has lots of eye dialect (and I read this out loud to my dad as well), though at least this variety of Malayalam was of course less foreign to me than the one in Chemmeen. This time, I was just really slow (I think this is the shortest novel I've ever read in Malayalam) and took eight whole months to read it (I was also busy with the epic poem and my grandfather's memoirs).