Hybrid RPGs? No problem
Horror RPGs:
-Koudelka (PS1) + Shadow Hearts 1 and 2 (PS2): Shadow Hearts in particular has a weird clock like mechanic for it's turn based battles
-Baroque (Saturn/PS1/PS2/Wii): Horror RPG + Roguelike, a lot of items with different effects to help you traverse the giant nerve tower
-World of Horror (PC and newer consoles)
-Off (PC, soon to get a remake for modern consoles)
Arena Fighter RPG:
-Custom Robo franchise (N64-DS): It's like RPG Armored Core, its insane this never took off
SHMUP RPGs:
-Undertale: not surprising that Undertale and Deltarune are kind of the only ones to do this type of combination
-Star Trader (X68000): The first of it's kind, but the combination of being JP only, releasing on an unpopular computer (X68000), being from a developer that's only now starting to take off from Ys (Nihon Falcom), and being crazy unknown means that this was never localized and it will be a while before it's fan translated
Gallery shooter RPG:
-Black Rock Shooter (PSP): Self explanatory, gallery shooter RPG mix
Beat em up RPG:
-literally every Yakuza game
-Sleeping Dogs
Dungeon Crawlers with a bit of freedom in movesets and classes:
-Etrian Odyssey franchise (DS/3DS/PC/Switch): Many different options to choose from (typically 7 classes). Can pick their portrait, name, which moves they'll start out with with the 3 skill points they get for free, the team name, a team of up to 5 (slot 6 doesn't actually exist, it's just there for a few moves and maneuvers to move the party around in the fight), and Etrian Odyssey V, the 1-3 HD remasters and possibly Nexus, the ability to change skin/hair/eye/etc color plus the voice. It's biggest downside is that the games are rough when it comes to getting EXP for benched members because of the 5 slot limitation, so it's recommended to go for Etrian Odyssey III (where there's a skill to address that very thing), Etrian Odyssey V (there's an item that also addresses that thing, but it's not obtained until floor 5 of Stratum 1), or the HD Remasters (adds a new difficulty called Picnic that can be used for easier grinding)
-The Dark Spire (DS): Etrian Odyssey minus the portraits. You don't see your characters at all, you gotta imagine them like you would in a D&D round.
-7th Dragon franchise (DS/PSP): Was Etrian Odyssey's sister series for a while so there's a lot of similarities. Most weren't localized, but fans translated them.
-Class of Heroes duology (PSP): I know nothing about these games
-Mary Skelter franchise (PS Vita/PS4/PC): I will not spoil myself on these games
Horror themed Monster Catcher:
-MegaTen franchise: Come on, this wasn't a surprise either. Type affinity, buffs, debuffs, items and smart strategy will get you through many boss fights, especially Nocturne onwards. Persona is a bit more lenient on you, but they're still SMT games, don't underestimate them either. Unique because good luck finding a non Atlus RPG focusing on normally insignificant shit like buffs and debuffs.
Category of it's own:
-Ore no Shikabane o Kete Yuke, or Oreshika (PS1/PSP): RPG all about sending your entire lineage into battle to eventually fell a demon who cursed the entire bloodline with a lifespan of like 3 years
-Ihatovo Monogatari (SNES): Barely even an RPG, it's more a life sim/puzzle game
-R Type Command (PSP): An SRPG spinoff of a SHMUP? IDK this game is weird
-Madou Monogatari 1-2-3 (MSX/PC-98/Genesis/SNES/PC Engine/Game Gear/Saturn): Dungeon crawlers with the distinct selling point of not having a centralised combat system, instead relying on combos like Street Fighter to execute different attacks. I don't know anything about what's in the most recent entry though, no one's really talked about it on Youtube either.
Whatever the fuck is going on with Mother:
-Earthbound (SNES): rolling HP bar that guarantees your survival against fatal attacks if you're quick enough
-Mother 3 (GBA): unique Nintendo patented rhythm game like system where hitting enemies to the music will net you an up to 16 hit combo
Mother 1 is a typical RPG, ignore