Now, I don't want to sound like I'm insulting Dungeons and Dragons. It was the first major RPG, and has kept pace with the industry over the years. However, no first attempt is perfect, and a few of D&D's imperfections have influenced RPG design longer than they should. This is my short list of RPG tropes that have outstayed their welcome.
I'm not sure who first thought, "Hey, what if some players had to start with characters that were randomly worse than those of their friends?" but I'm surprised at how enduring the practice has been since then. Fortunately, most systems are finally edging away from this method. Even D&D has largely abandoned this concept, listing it as one method among several in 3rd Edition and an openly-discouraged option in 4th.
I understand that the concept has some visceral and nostalgic appeal keeping it alive, but it's such stunningly bad design that it really needs to die out. Rolling dice for hit points at each level is a particularly egregious variant.
2) Overcomplicated Experience Systems
Let's look at a quick sample of the 3rd Edition D&D combat experience rules: First, look up the challenge rating for each of the opponents faced. Then, use these to figure out the total challenge rating for the encounter. Consult a table comparing the CR of the encounter and the average level of the heroes to determine the total experience for the encounter, then divide that experience between the heroes evenly. And even though you've already scaled the experienced based on the heroes' level, it's set up so that each level requires exponentially more experience to advance.
Now let me suggest an alternate system I came up with off the top of my head: You get one point of experience for overcoming minor obstacles or encounters, three points for difficult ones, and five points for defeating important opponents or completing major campaign goals. Characters level up every 20 points.
I get that there are reasons to use systems more complicated than the one I suggest, but lots of games still use ridiculously complicated systems that require multiple tables, multipliers, and scalers. These systems are fine for computer and console gaming where the numbers are crunched automatically, but are overcomplicated and completely unnecessary for tabletop RPGs.
#3) Loot-Based Economics
My first two points are easy targets - anyone can attack bad game mechanics. Looting is much more of a sacred cow, fundamental to many game designs. "Kill monsters, steal treasure," is one of the basic principals of the fantasy RPG.
In my experience, though, few things hurt drama or willing suspension of disbelief as much as looting corpses. It makes perfect sense in certain circumstances (post-apocalyptic settings, killing dragons with treasure hoards) but you don't see many books or movies where the heroes rifle through the pockets of every mook they kill.
Looting is a fine concept for "gamist" systems that aren't trying to emphasize storytelling or realism, but it has been far too widely-used in RPG design. Noble paladins shouldn't have to strip-search fallen opponents for loose change.