http://mp4nation.net/blog/2009/03/linux-ported-to-onda-vx747-jz4732-base...
Should be able to be ported to anything related to the JZ4732/40 chipset, including the A320.
Just having a GUI would be an issue, but It seems like a fun hacking project.
 
Then again, I'm sure you could compile lxde or xfce for the mips architecture, and possibly use the dpad as a mouse. (although using any desktop environment at QVGA is a frightening prospect at best)