The camera can rotate images by changing the exif data within the photo to say what orientation it was taken. This manual setting is set to 0, or 90 degrees so that when it is opened and edited it is the right way up. this ensures that the quality of the picture is the best, since the file does not have to be rotated and re-saved. Unfortunately Windows seems to completely ignore the orientation of the image.
Vista presents this portrait as a landscape, if I rotate the graphic and re-save it, it will open incorrectly on computers that understand orientation properly. Besides quality will be lost if I do this with a jpeg, since every time you save a jpg you get one generation loss of quality like ye olde video tapes.

Windows 7 shockingly still has this bug.

Mac here is displaying the same image, no surprises, everything works fine and I get a useful indication of the orientation data. Interestingly detected as TIFF data, which is what could be throwing Windows. Mind you, the Mac works whatever you throw at it.

Ubuntu does an excellent job of interpreting the orientation data.

My conclusion is possibly a tad extreme, but I don't think I could trust any version of Windows to give me consistent results in the realm of photography.
To be fair to Windows the fault probably lies with Fujifilm, except this does not explain why OSX and Ubuntu seem to work fine with every image I have thrown at them from many cameras. The issue can be sorted out by editing the orientation tag with a windows application.