Indie game storeFree gamesFun gamesHorror games
Game developmentAssetsComics
SalesBundles
Jobs
Tags

Issues with the PSP version

A topic by pierrelouys created Dec 12, 2020 Views: 711 Replies: 16
Viewing posts 1 to 11

Hello there, Superfury!

I've been looking for a good DOS emulator for the PSP for a while, so I was really glad to stumble upon your project. However, I couldn't get it to boot on my PSP 1000 - it runs out of memory. Perhaps it is incompatible with this model? 

I also tried to get it to work on the PPSSPP emulator, by setting it to emulate a PSP 2000, which has double the RAM. It would boot in this case, but even then, I couldn't get it to reach the BIOS - all it would do was let me change the internal BIOS settings. I also tried some older builds you posted on Vogons, but the result is the same (except, it won't even show the internal BIOS page).

I tried to stick with the most basic tweaks to the settings - 8086 CPU, XT architecture, and CGA graphics. I also tried ET4000, to no avail. I got all the BIOS files from the website linked from your Bitbucket wiki (except for the CGA BIOS), and the same configuration works fine in the Windows version - it boots into BIOS and reads floppy disks correctly. I'll be thankful if you can help me out.

PS: sorry for posting this in the release comments as well, just trying to get the message to reach you, since Itch apparently doesn't have PMs.

Developer (14 edits)

Does the app log anything inside the logs directory in the emulator folder?

I usually test it once in a while on JPCSP (2000 model) and on my PSP-2000. They both report the same issue only, mentioned below.

It should be able to run with minimal settings afaik. It just still has issues(weirdly PSP-only) with restarting the emulator, crashing the app. Haven't managed to find the cause yet, though. Everything looks fine on other platforms. Even the memory allocation module of the app reports that all data is properly deallocated during reboot. Perhaps some kind of memory corruption?

Edit: JPCSP confirms it. The app is still running, being at some function that delays seemingly infinitely. The screen isn't rendered (yet) on the PSP-1000. So it's either short on memory while running or short on memory before the display rendering is ready to use...

Btw, there's also the issue tracker for these kinds of things(it's within the links part, together with the manual).

Edit: Enabling the logging of the emulator initializing and finishing and disabling the encrypted prx instead of plain unencrypted ELF executable causes all those issues to magically disappear it seems? So the issue is somewhere in that?

Edit: Disabling the logging of the emulator initializing and finishing has the same effect.

So the issue is somewhere with the prx or it's encryption?

Edit: With the prx being compiled (both unencrypted and encrypted) instead of properly running, with the unencrypted version it gets out of memory errors!

Edit: Encrypting it while setting the heap size to keep 1024KB left for the prx to properly execute seems to fix the reboot and application initialization problem itself.

It seems that now, only when pressing select to open the settings menu while the emulator is running on the PSP and the display has the VGA screen rendered at least once on the frame buffer(it's allocated) somehow causes it to run out of memory?

Edit: Next, improving the thread manager to adhere to the SDL specs(that it needs SDL_WaitThreadEnd). Now the main function once again crashes when rebooting the emulator? Hmmm...
Edit: That was caused due to improper cleanup of the thread deletion function(waiting for it to end but not properly deallocating the entry, causing it to be reused with an invalid pointer later on).

Edit: Hmmmm... With those latest changes, it seems to somehow hang when quitting the application somehow (using the home button)? Perhaps something to do with the locking used?

Edit: After fixing the thread cleanup function to properly clean up the related thread data by waiting for the thread to quit, now there's only 1 thing left: when quitting the emulator using the home button, the emulator somehow starts hanging with the new threading setup.

Edit: Somehow related: found that the threading lock wasn't initialized during emulator initialization.
Edit: Fixed some locking issues with the new thread waiting for finish.

Edit: It no longer locks up now, but somehow it's corrupting something when exiting the game and the latest improvements of the thread manager are used? It only seems to happen if the rendering window is drawn? Before the video output is drawn, quitting the application (using the home key) happens without any errors.

Developer (2 edits)

So with the latest commits, the only error still remaining is that the quitting of the application somehow crashes the app when it tries to release some data on the stack(a free function call), apparently.

Unfortunately, I currently have no clue why this is actually happening. All other platforms still run without any visible issues.

Hi again, and thanks for the reply.

The Logs folder remains empty no matter what I try - I changed the debug settings on and off, but nothing is ever logged.

As a further test, I downloaded the most recent version and tested it with JPCSP (set to PSP 2000) - still only getting the internal bios, and a black screen after that. I tried discarding my settings file and starting over, making a single change to the default (pure CGA instead of VGA), but the result was no different. The same configuration works fine on the Windows version. A test with the ET4000 rom yields the same results.

Since JPCSP works for you, and the settings are (nearly) the default ones, the only option that I can think of is that the CGA/ET4000 bios files that I found are incompatible with the PSP for some reason. Have you tried your configuration on PPSSPP?

As an aside, I managed to recover the source code for Dosbox PSP and uploaded it here - it compiles and works, but the keyboard is pretty broken. If you felt like adopting the project it would be the greatest thing.

Developer

The bugfixes I've mentioned I made are not released on this site yet. They're only available right now when you download the source code and compile it yourself(the base PSPSDK 10.0 is all that you'll need).

When the SDK is installed(C:\pspsdk in my case), on Windows you can then run remake.bat in the UniPCemu folder, which will compile the app. Using Linux with a PSP devkit you should also be able to compile it, albeit running "make psp build" (without quotes) directly without the batch wrapper.

It's easy enough to compile for the PSP, though. Just download the PSPSDK 10.0 and add the missing encryption package. You'll also need the prx encrypter pack from gbatemp.net (google for it). Then just run the batch file on windows(or make command from Linux compatible) to compile the eboot.

Developer

The latest bugfixes, which includes the PSP bugfixes, are now out on itch.io! Take a look at it if you're still interested.

Hi Superfury,

it's good to see that you're still maintaining the project, but the issues I had six months ago still seem unsolved. In the meantime I bought a PSP-2000, so it's definitely no longer a matter of memory limitations. 

I did a test using the same approach as last time - using files that are proven to work with the PC version of UniPCemu. This time, it was the PC-XT bios 3.1 linked from the Itch page, and forcing CGA mode (so that no additional files would be required). I changed no other settings, other than to display the clock speed.

Again, this approach works brilliantly on PC - and does nothing on PSP. The CPU clock speed drops to 0% and the screen stays black - even after minutes of waiting.

Developer

Just ran the current commit version (which has some optimizations and speed improvements (up to 75% more speed, purely optimizing DMA clocking and non-rendering audio channels(mostly the MIDI synth's 24 idle note channels)).

When in cycle-accurate CPU mode, it's indeed very slow (constant 0%, although running). It's has a green outline, so it's actually running (but at less than 1% speed).

In IPS clocking mode, it's faster (more than twice the speed) as measured on an i7-4790K (~35% in cycle-accurate, ~80% at IPS clocking mode(at 315KIPS with it's default setting used).

In IPS clocking mode, I do see the speed percentage going from 0% to 1% and back every few seconds. So it's definitely running, but very slowly.

Can confirm, even with the release version I can see the CPU speed hovering at 0-1% when set to IPS clocking mode. So it is running. After leaving the PSP alone in a corner for several minutes, it does display the beginnings of a launching BIOS.

Other than switching to IPS, are there any settings that I can change to get it somewhere close to being usable?

Developer (1 edit)

Well, you can lower the cycles setting to make it run more realtime(less cycles = more realtime).

But that will come at the cost of running software of course slowing down, as the amount of instructions each second (in emulated time, not realtime) becomes less.

The main issue is that according to most profiling I did on Windows builds, it's spending most time on memory accesses alone (RAM and ROM memory). I'm trying hard to optimize it to get faster, but it's proven difficult because no dynarec of any sorts is implemented (it's about ~20% CPU emulation(of which about only memory accesses itself being the main cuprit for that), ~20% video card emulation and 60% of only the other hardware and timing overhead).

It's running better on Android (5%) and i7-4790K Windows (~20%), but I have no idea what to change atm to make it more fast right now. Most memory accesses are already reduced to single 16/32-bit accesses where possible, but most of it seems to be prefetching only (filling up the Prefetch Input Queue to contain the maximum instruction length each instruction). Although said overhead is less on cycle-accurate mode(due to not loading unneeded bytes from memory most of the time), it has it's own optimization difficulties (due to ticking hardware and CPU multiple times during a instruction (the amount of CPU cycles for an instruction)).

Disabling some of the emulated hardware should theoretically free up some speed for the CPU to run faster(if it's enough), but the CPU is mostly ticking way more than the hardware anyways(all but VGA and CPU always ticking at 14.31818MHz base intervals, with the VGA depending on it's state(25/28MHz on VGA, 14.31818MHz on CGA, EGA, MDA clock crystals, even higher on SVGA when setup(up to Dosbox's ratings of the Tseng chips)) and CPU acting as the base timing for all hardware (divided it up into video card and 14MHz(1MHz for Sound Blaster, ticking at 14MHz base clock division) depending on it's speed setting(the IPS cycle setting).

Dosbox PSP does have a dynarec - code is here if you want to have a look.

Developer

Well, the issue with dynarec is that it completely destroys any cycle-accuracy. I want to keep the emulator capable of doing that(and Dosbox doesn't have that issue, as it runs instructions in blocks anyway).


Currently the main issue preventing speed is the sheer amount of memory and ROM read/write operations itself. So that means that the BIU and mostly instruction fetching itself (which is reading memory most of the time) is the main bottleneck. The memory reads themselves are already optimized to be only a single call when the address is byte/word/dword aligned(it then uses direct memory access to read/write data to/from the CPU cache or write buffer(for writes only) using a simple pointer dereference). Unaligned dword data is handled as words if possible, bytes otherwise.

But even with all that, the profiler still reports the memory accesses themselves being the hot path.

Developer (2 edits)

Is there a green or red outline on the 0% text? If it's a green outline (and using the select button brings up the settings menu), then the emulator itself should be running(although at less than 1% speed).

The latest commits for the next release (already in source control) does improve the speed somewhat (on PC about a 75% speed increase by simply not rendering and parsing(filtering) non-rendering sound channels (which are pretty much all 24 MIDI rendering channels that aren't rendering any audio yet or anymore).

I did notice (using JPCSP, don;t remember the version off the top of my head) that it was in fact running last time I tested it, just very slowly.

Edit: 

Somehow it's having at least some difficulty running even somewhat running, at least in cycle-accurate mode.

IPS clocking mode starts the app almost instantly (save for the disk accesses, which are slow anyways due to the memory stick being used).

Cycle-accurate mode seems to be way slower at that. That seems to have been about 5 minutes just starting up the emulator using it's first few instructions somehow (those are only about 10 instructions to start the whole thing, which is weird because that happens only only when initializing the emulator and displaying the yellow text (it's an emulator escape wrapped in a simple call wrapper which starts the actual thread). That's observing the delay between the first and second memory stick accesses(listing the ROM directory, loading all avaiable ROMs). This happens twice during startup: once when loading the emulator initially, and another time when resetting it to start the proper emulation (unless the internal BIOS is used, which foregoes the second step).

That's pretty weird.

Edit: Just let it run in cycle-accurate mode for 2.5 hours... It checked a whopping 160KB of memory when posting!

So both modes are running. It's just that the cycle-accurate mode is somehow really ridiculously slow on the PSP. IPS clocking mode seems way faster.

Developer (3 edits)

After tinkering about with the memory allocation functionality UniPCemu uses (to keep track of used memory allocations), I've managed to shrink it's current memory usage for those to only 5% of what it used previously. So from 2240KB to only 450KB, freeing up another 1.8MB for user memory or video memory on the PSP :D Although it's only in source control right now, I'm probably making a release version of it soon (together with the other improvements for speed, packet server(Seperated Pcap library support improved on Windows(using Dosbox based dynamic loading of the DLL, allowing for it to become optional unless required for the packet server (when enabled)), now supporting all possible line break formats (Mac, Windows, Linux) in it's inputs, ARP support added for the static IP addresses (allowing for custom addresses outside the DHCP space to be used for each logged in client, disconnect crash fixed)), Sound Blaster IRQ switched to IRQ7 (fixing Windows 9x detection of the IRQ) and various optimizations (up to 75% faster than the previous release so far, reaching it's old speed in certain Windows 9x cases again(Windows 95 at 20%+ realtime speed)))).

Edit: Improved the MIDI active sense a bit(locks).

It still crashes in the JPCSP emulator (null pointer dereference somehow)?

OK. I've changed a few little things on the common emulator side of things:

  • Changed the timer thread to wait 1 second before trying to check for it's termination through SDL 1.2 after terminating it (this is only called once when restarting the emulator environment, which shouldn't be much of an issue).
  • The timer enabled flag would be cleared in that case now before terminating the thread instead of after (ommitting the unlock and relock for this).
  • On SDL 1.2 it won't wait for threads to end using SDL_WaitThread if the thread itself already reported itself having terminated.
  • The order of cleaning up the emulation state has been changed in the main thread: First, stop the timers using a soft-reset (keeping the thread alive if it's present). Then, terminate the emulator if needed (which also saves any changed to the machine state on disk (Settings file)). Finally, stop the timer thread itself. (Previously it would stop the timer thread in one call, performing emulator termination afterwards). After all that, the normal cleanup sequence starts (terminating threads followed by SDL cleanup and remainder cleanup (terminating SDL input itself, terminating audio, terminating video, terminating PCap(on available builds), terminating the app itself))). The steps from the termination of the SDL input and onwards are skipped over when soft-resetting the emulation using the settings menu (which seems to run fine on the PSP all the time somehow, even on current releases of UniPCemu).

Only the first few steps (stop timers, finish emulator) are switched now. It will first stop the timers themselves, then emulation (causing the settings to be saved properly) and only after that try to terminate the timer thread itself.

Somehow, I notice that during the termination of the timer thread itself, something seems to trigger the exit callback or actually calling the exit game method directly on the PSP (the JPCSP logs say it's from the "update_thread" thread)?

Developer (4 edits)

Hmmm... Just 'imported' the SDL_main code from the PSP 1.2.15 branch diff into UniPCemu's SDL_main, then removed the SDLmain library and define from the LIBS and CFLAGS. So in that way, UniPCemu uses a copy of said code to initialize SDLmain on the PSP.

So now when testing it again, the main thread actually seems to call the sceKernelExitGame function instead of SDL's update_thread.

Now I still need to check if it's actually the main cleanup doing the termination.

So far it just looks like SDL's default update_thread in it's SDLmain library was interfering with UniPCemu's replacement handler (which lets the main thread handle termination properly instead of (what PSP SDL does) directly calling the exit() function to terminate the app).

And since the atexit also registers various structures UniPCemu allocates itself(it's custom memory management handling, cleanup of structures), perhaps it was conflicting with what the main function was also doing at the same time (those final steps)? Probably the threading having memory conflicts?

Edit: Just temporarily for debugging enabled the logging and behold: the application now properly handles it's termination past the last few steps with the SDLmain custom implementation for the PSP builds! :D

Edit: Just tried it on my physical PSP-2000. It seems to run without issues now, including quitting the app! :D

These fixes will definitely be in the next release, once it's ready!

Developer

The changes are now released with the latest version of UniPCemu on itch.io!

Is the app working without issues now?

plz help me to make a PSP DosBox Hack => PSP DosBox Hack Speed Version Need Help ! \ VOGONS