For babby's first RPGs can't go wrong with Imageepoch developed RPGs. 7th Dragon 2020 & 7th Dragon 2020-II, Fate/EXTRA & Fate/EXTRA CCC, Black Rock Shooter: The Game, Last Ranker and Sol Trigger are all pretty good. Same with all tri-Ace PSX remakes on PSP: Star Ocean: First Departure, Star Ocean: Second Evolution and Valkyrie Profile: Lenneth.
Then you can move to Eiyuu Densetsu/The Legend of Heroes series, especially the Sora no Kiseki/Trails in the Sky series tho be warned that all of them are connected with each other so you might be overwhelmed if you wanna experience them all.
Summon Night 3, 4, 5 are also pretty good tho sadly only Summon Night 5 had a proper English release.
Also Department Heaven series for sure; Yakusoku no Chi: Riviera/Riviera: The Promised Land, Yggdra Union and Gungnir are still some of the most innovative and fun experience RPGs. The rest are either still Japanese exclusive (Blaze Union and Gloria Union) or in case of Knights in the Nightmare, absolutely polarizing bad.
Yarudora Portable series are pretty much games that actually played like TV anime show with CYOA choices, and all of them are real good especially two that had already fantranslated: Double Cast and Sampaguita.
The rare anime adaptation games are also surprisingly pretty good on PSP, especially those that are already fantranslated. Suzumiya Haruhi no Yakusoku/The Promise of Suzumiya Haruhi,
Toradora! Portable, Puella Magi Madoka Magica Portable, Ore no Imouto ga Konna ni Kawaii Wake ga Nai Portable ga Tsuzuku Wake ga Nai/My Little Sister Can't Be This Cute In This Portable Continuation, Strike Witches: Hakugin no Tsubasa/ Strike Witches: Silver Wings, and Shinseiki Evangelion: Koutetsu no Shoujo 1 & 2/Neon Genesis Evangelion: Girlfriend of Steel 1 & 2.
For Japanese-exclusive or incomplete fantranslations, I'd suggest Fate/tiger colosseum! and Fate/tiger colosseum! UPPER (fun Fate series party game spinoff), Shining Ark, Shining Hearts & Shining Blade (fun Shining Force RPG branch series) and Tokimeki Memorial: Forever With You and Tokimeki Memorial 4 (two most legendary dating sim games of all time).
tl;dr You really can't go wrong with most RPG/anime-like games on PSP, they all look and play pretty good in general.