The answer to your question is simple: look at TES.
Redguards exist, right? Dunmer exist, right? Both are considered highly in the context of the setting, right? So why would Orsimer be a racist caricature (if handled less well than Bethesda has done so far)?
Because of the coding. The same way that any eugenicist species in a scifi setting is automatically coded as Nazi Germany, certain traits make “this race/species our stand in for real world group A”.
Want an on the nose example of that that’s relevant to very recent events?
Goblins and house-elves in Harry Potter are unquestionably racist Jewish and Black caricatures. Harry Potter takes place in the real world, so there are Jewish and Black people.
But the coding is undeniable: goblins are greedy old bastards with long hair and big noses who control the world from within a banking system.
House elves are born-slaves who cannot remember not being enslaved and actually prefer the slave life.
These are ANCIENT racist caricatures. Putting a different name on them to tie them to a fantasy race does NOT separate them from their inherent racism.
More subtle attacks are involved with orcs in D&D, but consider momentarily:
Orc art tends to be of beefy, solemn looking hominids with skin that is darker than white person skin, and Black or Asian features.
Orcs are violent, dimwitted, steal everything they own, and are more a danger to each other than to the adventurers who slaughter them by the hundreds.
Orcs (or related goblinoids, especially hobgoblins) are described as having ape-like features, as being hairy, and are depicted with long monkey-like arms.
Mating with an orc is almost always rape, and always produces a tragic youth who doesn’t know one or both of their parents, is dumb and is given to violence.
It’s fucking racist.