Archive for the ‘Development’ Category

June 12th, 2010 by nkeynes
ISO9660 support update
Posted in Development

I recently discovered that there is in fact a very nice ISO9660 library out there already called, oddly enough, libisofs. I’m not sure why I didn’t come across it earlier, but in any case I’ve ripped out my half-baked ISO support and added a dependency on libisofs instead. It looks to be available prebuilt for most systems, so hopefully no major dramas there.

The on-the-fly ISO-building support is in now as well (mainly thanks to the above), at least for executables. You can now load binaries from either the gdrom dialog (which packages it into a MIL-CD disc image), or the ‘load binary’ dialog (which does what it always did). The command-line is also changed slightly – “lxdream mygame.elf” loads the program as a disc, and “lxdream -e mygame.elf” executes it directly.

Additionally, loading “binary” files (i.e. as generated from “sh-elf-objcopy -O binary”) is now supported (seeing as it turns out that this wasn’t working previously). Since there’s no reliable way to identify them from content, they have to end in “.bin” to be recognized though.

So with that out of the way, I think I might go back to poking at the renderer the next time I have a few free moments. One thing at a time…

February 25th, 2010 by nkeynes
CDROM work-in-progress
Posted in Development

The (hopefully last) major refactor of the cdrom host subsystem has finally landed in trunk – this shouldn’t be particularly user-visible at this point (unless I’ve broken something, which is entirely possible), although it does fix a few long-standing bugs that no-one ever ran into. Mostly this is to make things more flexible, so that I can add features without it becoming an even more tangled mess.

The first of these is the ability to boot from disc without actually needing the Dreamcast boot rom. This is in now, and is working for dclinux and (I think) most homebrew. As far as I know it doesn’t work for any commercial titles at the moment though. As a nice side-effect it boots faster too (as it doesn’t have the flashy logo  screen).

The other piece of work that this supports is on-the-fly construction of cdrom images – for example, taking a binary program and encapsulating it automatically in a bootable image. Also of interest is support for SBI “images”, which are basically filesystems-in-a-zip. This also makes it easier to add image formats like BIN/CUE and BIN/TOC, so I’ll probably add those in the near future as well.

In unrelated news, OS X 10.6 has been giving me a lot of grief – exception unwinding support seems to be subtly broken (worse, in a way that’s hard to detect with a simple configure test). I wasted far too much time trying to track down a memory corruption bug that eventually turned out to be coming from the unwinder as well *sigh*. So for the time being I’m going to settle for building on 10.5, and hopefully it’ll work on 10.6 as well. Incidentally, I’m intending to drop support for 10.4 for the next version unless someone screams really loudly – trunk hasn’t actually built on 10.4 in quite some time, and no one has complained  yet…

What’s new

  • CDROM host subsystem finally properly separated from the Dreamcast bits. Much better now
  • BIOS-less booting of discs. Most homebrew should be able to run now without the actual boot rom.
  • New -b command-line option to run without the BIOS even if it’s configured
July 5th, 2009 by nkeynes
Mailing lists
Posted in Development, Website

It’s been pointed out to me that lxdream currently lacks mailing lists, and that it might be a good idea to have some. I mean, sure we have the forums, but they aren’t really as convenient as email, are they? So since my esteemed webhost actually has mailman setup and ready to go, I’ve gone ahead and setup the standard lists with lxdream-users, lxdream-dev and lxdream-commit – click here to go to the subscription pages. Mercurial commits are already going to lxdream-commit (albeit HTML-only at the moment, using CVSspam for the notifications)

You can also monitor the repository through the RSS feed at – whichever you prefer, but the mailing list has more detail (ie full diffs).

June 15th, 2009 by nkeynes
Been a bit quiet around here
Posted in Development

Not much really got done last month, seeing as I was away for nearly all of it, but I did get a few small tidy ups done. The new translation core is definitely on hold now for a while so I can get some other things done – I’ll get back to it at some unspecified point in the future.

So… I suppose what everyone’s waiting for: There will be a 0.9.1 in the next few weeks (also known as the “what do you mean we haven’t had a release in 8 months?!” release), which will be more or less current svn trunk + one more feature I’m working on at the moment.


  • Wahrhaft sent in a patch to add hotkey support, quick-save-states, and an LIRC input driver, thanks!
  • Refactored the gdrom support module to be a lot cleaner, and make it easier to add other MMC-based native drivers. I think a fixed a few bugs in the process as well.
  • Added ability to build drivers that require 3rd-party libs as dynamic objects rather than statically linked – this should make binary packaging a bit saner. On by default for now, configure –disable-shared to turn off.
  • GTK: Added GNOME desktop entry for more convenient desktop integration
  • OSX: Added fullscreen support (“er wait, the OSX port doesn’t do fullscreen?”)
March 24th, 2009 by nkeynes
GDB debugging support
Posted in Development

I was distracted by Anthony Green’s Moxie blog this last week and a bit, specifically by the bit about qemu gdb support. This seemed like a really good idea, not to mention being fairly simple to implement, so it’s in now for both the SH4 and ARM. The actual debug support just hangs off the existing debugging framework, so the work needed was really just to implement the protocol.

If you want to have a play with it, run lxdream with -g <port> to start an SH4 GDB listener on the given TCP port, or -G <port> to start an ARM GDB listener. Or both for that matter, although gdb is likely to be confused if you actually connect more than one debugger at a time. To note the obvious, you’ll need to build an sh-elf-gdb or arm-elf-gdb (respectively) for this; your system debugger is just going to get confused. Also worth noting is that GDB tends to default to big-endian if there’s no file loaded – you need to explicitly force it to little endian (“set endian little”).

It supports all the usual bells and whistles except watchpoints, which I’m looking into at the moment. They should be reasonably straightforward to implement now, and will have zero cost when they’re not being used. And as a nice side-effect of all this I can finally do a full implementation of the on-board UBC.

I probably should have done this a long time ago – It’s nice to have a full symbolic debugger without having to actually write one from scratch ^_^

March 1st, 2009 by nkeynes
No Feb release
Posted in Development

There’s not going to be a February release as originally scheduled – there’s just not enough ‘stuff’ ready to make it worthwhile in my opinion, and anyone looking for the latest has probably already pulled it from SVN. At the current (depressingly slow) velocity, there may be something releasable in April. We’ll see – I need to make sure the new SH4 translation core is solid, and make some decent headway on the PVR2 performance problems first.

Update-wise, we have a new SDL audio driver now, thanks to Wahrhaft, but otherwise not much of note to report at the moment. However, this might be a good time to note that there’s lot’s of other work that would be good to see done, but I’m simply not going to have time for any time soon – so if anyone out there is interested in contributing in any way, please let me know. There are some obvious areas that might be a good place to get started with lxdream -

  • OS X port – The base port is fairly stable, but there’s a number of outstanding issues, and the UI could no doubt be improved.
  • GTK UI – for that matter, the GTK UI isn’t very pretty either and could use some good UI design attention.
  • Audio subsystem – Current audio is almost obnoxiously bad, with plenty of bugs and missing features. Fortunately MAME has a fairly complete implementation that could be used for comparison and testing.
  • Translations – we have partial translations to german, spanish, italian, and portugese so far, which need to be updated for 0.9, and it would be great to support more languages as well. There are still some untranslated strings that need to be fixed as well.
  • SBI ‘image’ support, not to mention the many other bugs/enhancement requests in the issue tracker.

And of course, anyone interested in working on improving lxdream in any other way would be eagerly welcomed too ^_^.

February 10th, 2009 by nkeynes
Timing and update
Posted in Development

I’ve had some high-level performance numbers kicking around for a while – give or take a few percent they’re fairly consistent, at least for the work-loads that I’ve been profiling. So, expressed as seconds of real time per second of emulated time, the runtime looks something like this on my machine with current svn head (0.9 was quite a lot slower):

  • 695 ms – 3979 ms Rendering and display
  • 264 ms SH4 Translated code
  • 76 ms SH4 mem read/write
  • 55 ms SH4 support code
  • 64 ms AICA/ARM
  • 57 ms Tile accelerator
  • 45 ms Render Scene parsing
  • 9 ms Miscellaneous

SH4 total: 395 ms, All non-rendering: 570 ms, Grand total: 1.265s/4.549s. The two rendering numbers are for 2 otherwise fairly similar machines, one with a 9800GTX and the other with an 8600M GT (I’m sure you can guess which one is which). The times are also with TLB off (which still has an appreciable although lesser impact than previously)

By way of comparison, I’d originally budgeted for CPU along the following lines (although I was also targeting a much slower machine than I have now as well):

  • 50% SH4 emulation
  • 25% rendering
  • 25% everything else

Rendering performance is quite obviously atrocious at the moment and needs a lot of work, which may or may not make it into 0.9.1 at this stage. At the moment though I’m still focusing on SH4 performance – while we’re technically under budget already, one has to bear in mind that a) the SH4 is currently underclocked by a good bit, b) there’s a few things I will need to add for accuracy that are likely to hurt performance (possibly severely in some cases), and c) Given the rendering problems I’m now aiming for 50% rendering / 35% SH4 / 15% everything else, as this may be a more achievable target.

In the meantime, the new multi-pass translator is slowly taking shape, having gone through a few fairly major revisions by now. I’ve ended up with a fairly simple low-level 2-op IR that (mostly)?maps directly to the x86[0], plus a few ‘macro ops’ to express the handful of SH4 instructions that don’t have a simple representation on x86. I’m currently working on the x86 codegen backend, which should be ‘done’ sometime this week, so the whole caboodle may actually be fully working by the end of the month after all.

[0] Having said that, it’s not at all x86-specific, and it should be a heck of a lot easier to add other targets than with the old codebase. You know, should anyone actually want to do that…

January 28th, 2009 by nkeynes
Testing with Intel’s icc compiler
Posted in Development

Out of sheer curiousity, I thought it might be worth seeing how icc performs on lxdream – short answer, not too shabby at all. All tests otherwise with the same command options, best of 3 runs:

Compiler 5-second core runtime Improvement
gcc -O2 3.10s N/A
gcc -O2 -fprofile-use 2.96s 4.6%
icc -fast 2.96s 4.6%
icc -fast -prof-use 2.73s 12%

Profile runs using profile generated for the same test. 5% is kind of meh, but 12% on the icc profile build… ok that’s pretty nice. I will probably have to look at generating some decent general purpose profile traces for production builds

In any event, I’ve added support for building with icc, for the benefit of the 2 people who actually have it ^_^.

Otherwise I’ve finally got some very basic UTLB test cases in now, and fixed a number of bugs that turned up – it seems to be at least as stable as the old version was by now (which actually still had a few bugs too incidentally…)

January 14th, 2009 by nkeynes
Memory system rewrite
Posted in Development

The memory system rewrite is merged now – there are a few things I’m not completely happy with yet, and the old page_map isn’t quite gone completely, but on the whole it’s simpler, faster, and much more consistent. More importantly perhaps, UTLB translation is now _very_ cheap (3-instruction overhead[0] for OSes using the typical 4K page) – linux now boots and runs at full speed on my systems. There’s probably a few lingering issues and I’m still working on a good test suite for it[1], but most bugs are likely to be in things that never worked before anyway.

I also have some work-in-progress on the operand cache (nominally the original reason I started doing the rewrite…), but it’s still showing a bit more of a performance hit than I would like (10-15%). So currently I’m thinking this will probably wait for the next version before being fully integrated and finished. It does need to be done eventually though for correctness reasons, since the SH4 doesn’t ensure cache-coherency in hardware.

In any case, once the MMU tests are done I’ll get back on the translator upgrade. It’s looking at this stage like 0.9.1 will end up being almost purely a performance release, but since it should be at least twice as fast overall as 0.9, no one is really going to complain about that, right? ^_^

[0] We might be able to special case sdram access and get that case down to 0 instructions, but leaving that aside until after the op-cache is done…
[1] Annoyingly enough, there doesn’t seem to be a good way to recover from TLB multi-hit resets on the DC, which makes it a little hard to test that aspect of things… Even more annoyingly, the DC BIOS _does_ vector manual resets through 0x8c000018, but not any other reset.

December 9th, 2008 by nkeynes
I am Jack’s complete lack of update
Posted in Development

November has, unfortunately, been very busy with other matters, so lxdream hasn’t really gotten much attention lately – the little that I have gotten done has been rather scattershot. Right now, at least, I’m currently hacking on the memory system (ie, implementing the caches and bus timing), and will eventually get back to the translator sometime after that’s working.

But just to scare any programmers reading this, below is my latest configure test:

AC_MSG_CHECKING([support for dirty stack tricks]);
void * __attribute__((noinline)) first_arg( void *x, void *y ) { return x; }
int __attribute__((noinline)) foo( int arg, void *exc ) {
    if( arg < 2 ) {
        *(((void **)__builtin_frame_address(0))+1) = exc;
    return 0;
int main(int argc, char *argv[])
    goto *first_arg(&&start, &&except);
    return foo(argc, &&except) + 1;
    return 0;
}]])], [
    $1 ], [
    $2 ])

For when longjmp just isn’t fast enough. Although one does have to jump through quite a few hoops to stop gcc from enthusiastically (mis)optimizing the test case…