gpSP workaround...
The emulator will hang every time you try to select Exit gpSP or enter into the Graphic and Sound Options. The emulator tries to initialize/overwrite gpsp.cfg file but it is missing from the gpSP folder (failed to generate, file error, etc.).
So download gpsp.cfg + game_config.txt and place it in the same folder as gpsp.dge and gba_bios.bin. I not sure if the game_config.txt database will be utilized in Dingux's gpSP, never test it.
And it seems like the gamepad input will ignore any mapped buttons while the other options will not save into the gpsp.cfg file.
Extra: About the gba Kingdom Hearts - Chain of Memories game, you don't have to copy a *.sav file to skip the freeze animation. All you have to do is, when the intro animation freeze up, go into the gpSP menu and do a savestate. Then reload the savestate to skip that error (the animations will glitch but you can continue to play normally afterwards).
Get the gpSP emulator here. As for the BIOS... Seek and ye shall find.
Note: There's an outdated game_config.txt in the gpSP source file, but I bundled the config from SCDS2 GBA EMU (game_config.txt) 1.1 instead.
Wha...
By the way, Earth Worm Jim (1st) plays better on gpSP than the native Gba. Will try the other games when I got nothing to do...
And to correct the sound (for almost any games), set the Audio buffer to 2048 bytes and then exit the emulator to save the settings. Reps to Stephanie at Dingoonity for this one :)

*Edit*
Nevermind. :)