This is an extended strawman argument with a heaping dose of the fallacy of the excluded middle. I went back and read the whole thing just to make sure I hadn't missed anything and, nope, I hadn't, he doesn't ever develop his rhetorical figure of the anti-racist intent on "cancelling" him beyond the thinnest caricature.
("Cancelling" is a real bugaboo with him, isn't it? He mentions it 11 times in 36 paragraphs. After a while it starts to feel like a plea, like he wishes anti-racists cared enough to "cancel" him when--as far as I can tell--most simply ignore him.)
I'm not going to go through his arguments point-by-point, but here are a couple that leapt out at me:
Seeing prisons as a racist conspiracy to confine black people is an absurd proposition. No serious person could believe it.
Has he really never heard of the conviction of Mark Ciavarella Jr? Odd, because it was kind of big news at the time.
They act as if pointing to the absence of wealth in the African American community is, ipso facto, an indictment of the system—even as black Caribbean and African immigrants are starting businesses, penetrating the professions, presenting themselves at Ivy League institutions in outsize numbers, and so forth.
Striking for someone who accuses his opponents of ignoring facts, he only briefly mentions the wealth and attainment disparities noted by every serious researcher who has ever compared these two groups before he dismisses them as unimportant. And then--striking for someone who goes on and on about how "culture" determines attainment--he doesn't even talk about the observed cultural differences between these groups and how or why they exist. Here's Tauriac and Liem's summarisation in their 2012 paper on the subject of how they manifest in a college context:
Yet, on the other hand, an emerging body of research has established that immigrant-origin Black students are often regarded by faculty and peers as being more intelligent, academically-motivated, and affable than U.S.-origin Blacks (Charles et al., 2008; Offoh, 2003; Tormala & Deaux, 2006; Williamson, 2007). Consequently, relative to Black immigrants, U.S.-origin Blacks experience greater academic stigmatization and may feel less supported and less comfortable navigating predominantly White contexts (Deaux et al., 2007; Williamson, 2007). These social adjustment difficulties often impair U.S.-origin Blacks’ academic performance and increase their vulnerability to processes such as stereotype threat and social identity threat (i.e., the fear of doing something that might confirm a negative stereotype and the concomitant psychophysiological arousal that undermines cognitive and interpersonal functioning; Deaux et al., 2007; Steele, Spencer, & Aronson, 2002),
Conversely, immigrant-origin Blacks come from (or in the case of second-generation immigrants, have been greatly influenced by their parents who come from) cultural contexts in which Blacks have historically held significant positions of political and social power and racialization was absent within societal institutions (Dodoo, 1997; Tormala Deaux, 2006). Thus, relative to U.S.-origin Blacks, Black immigrants may feel more comfortable interacting and building relationships with outgroup members and, consequently, may engender more support from White faculty and peers (Deaux et al., 2007; Massey et al., 2007; Tormala & Deaux, 2006; Williamson, 2007). This support may have a positive effect on immigrant students’ academic performance (Williamson, 2007; Gerdes & Mallinckrodt, 1994).
Tinto’s Model of College Persistence.
tl:dr: USAmericans (of all races) frequently harbour prejudices against US-born African Americans which they don't hold or don't hold to the same degree when it comes to Blacks born abroad, and this affects their behaviour towards them. Moreover, Blacks from abroad have experiences of Black-dominated societies which affects how they react to US racism. (The researcher John Ogbu has argued that voluntary minority immigrants are more likely to attribute the discriminatory treatment they receive to their status as “guests in a foreign land” and believe that the barriers they face are temporary challenges they can overcome through hard work, greater acculturation, and academic attainment. Needless to say, US-born Blacks have very different experiences and have drawn different conclusions about how "temporary" those barriers are and what the real limits on their attainment are.)
One last bit:
These racial disparities have multiple interwoven and interacting causes, from culture to politics to economics, to historical accident to environmental influence and, yes, also to the nefarious doings of particular actors who may or may not be “racists,” as well as systems of law and policy that disadvantage some groups without having been so intended.
I'm reminded of G.K. Chesterton's famous quote about skinning cats. Here Loury is the new theologian and the skinned cat is the abundant evidence that laws and policies have been adopted and continue to be adopted to intentionally disadvantage some groups. Again, I wonder where he gets his news from because deliberate GOP efforts to suppress the Black vote were major headlines all last year and in 2018. He's already willing to concede that there are structural factors at work, so why go out of his way to let policymakers off the hook for their racist actions?