24 March 2013 by Published in: rants 5 comments

Continuing my N-part series on “how not to do useful things” that tells you nothing useful about how to solve your problems, I present to you: how not to install Windows on an external drive on your Mac.

Imagine this: you operate on a tiny 120GB SSD.  You need to run Windows for some reason, a reason that renders VMs insufficient.  Therefore, you try to install Windows on the external drive.

Is this supported?

Of course not.  It came as some surprise to me, as a person who regularly runs OSX from various external drives in production, that it’s in Microsoft’s interests not to support it.  Worrying about piracy and that.

A word about “Boot Camp”

There’s a lot of misunderstanding about what “Boot Camp” actually is.

1.  A set of drivers that you install on the Windows side.  Dell or HP would just call this a driver disk, but you know, it’s Apple, so we need a marketing name.

2.  A glorified wizard for Disk Utility that creates partitions and makes bootable USB install media (if you have certain Macs)

3.  Firmware installed on every Mac.  These are these things called “SMC Firmware Update 2.7″ and “EFI Firmware Update 1.5″ that you get once in a blue moon.  It does a surprising amount of things.

So let’s talk about the firmware.  This is not your grandaddy’s BIOS image.  To whit: it probes your drives, contains lightweight video drivers, full USB, Firewire, and Thunderbolt stacks, configures the Bluetooth subsystem, unlocks FileVault2 disks, has fairly complete CoreStorage stack, a full cryptosuite implementation, configures the WiFi subsystem (including a full 802.1 a/b/g/n implementation with WEP/WPA/WPA2 etc.), complete FAT32 and HFS+ implementations, controls Lion Recovery, implements Target Disk Mode, and on newer Macs it does a complicated dance with OSX to implement Power Nap which supports, among other things, Mail, Software Update, Time Machine, and many more.  Oh, and it also has a crappy BIOS emulator that can boot Windows.

So a brief history of BIOS.  In the beginning, when your computer started, it loaded the first sector or so off a disk into RAM and then just JMPed to it.  And IBM saw it, and said that it was good.  And it was evening, and it was morning: the first day.

And Microsoft said: “Our kernel is a great kernel; and it cannot fit into a boot sector.”  And thus was born the Bootstrap Loader, which would fit into the Boot Sector and load the kernel.  And Microsoft saw that the Bootstrap Loader was good.  And it was evening, and it was morning: the second day.

But the people fell into a grave sin: they abandoned the way of their fathers and installed many pagan operating systems that their forefathers knew not.  And the Holy One came and confused their filesystems so that the operating systems could not understand one another.  Thus one spoke FAT12 and another FAT16, one needed a disk signature and another needed an Advanced Active Partition.  And each release needed a larger Bios Parameter Block than the one before; but the filesystems that the bootloader must understand continued to grow, and the maximum size for the boot loader became smaller.  Thus the software vendors said: “we shall have a boot loader for our boot loader.”  And they created NTLDR, and GRUB, and Syslinux, and they saw that it was good.  And it was evening, and it was morning: the third day.

But the people grew restless; for it was inconvenient to install Windows NT from 22 floppy disks.  And they could design an entirely new standard, but it would be inconvenient for the software developers to maintain two install methods for the OS.  So the El Torito standard was born, which allowed computers to be bootable with CDs by emulating floppy disks and without changing existing software, and keeping the existing 2-bootloader toolchain.  And Phoenix Technologies saw it, and it was good.  And it was evening, and it was morning: the fourth day.

And soon, the venerable floppy disk passed away.  Nonetheless, every conceivable boot option, to maintain compatibility, basically emulated floppies.  Intel introduced PxE (known to PC builders as the world’s most useless boot option) which contained a full TCP/IP stack the sole purpose of which was to download 1.44MB floppy disk images over a network.  El Torito was updated to support the El Torito boot process over USB-based CD/DVD drives; USB hard disks and flash drives were variously configured either to emulate El Torito emulating floppy disks, or to emulate floppy disks themselves.  And at the core of all of these boot options, to the present day, are a severely size-restricted boot sector that is constrained to only access 32-64Kb of memory and only use instructions available on the original 8086 processor.  Not kidding.

And lo, Apple wanted to build an Intel computer.  And it looked at the landscape of everybody emulating 1980s computers and determined that this was complete chaos.  Apple wanted to obsolete, not just the floppy drive, but also the optical drive; not just the optical drive but also the spinning hard disk; not just the hard disk but the entire computer form factor.  (And they also had competing replacements for things like PxE that they did not want to port to the 8086.) So they collaborated with Intel to create a new standard, called EFI, which we will discuss in some detail later in the article.  But for now, know this: it was very, very incompatible with the way everybody else did things.  Thus for Windows support, they shipped a very small BIOS.

But this BIOS was not the BIOS of 2001 with twelve kinds of USB support for emulating floppies.  Rather, it was the minimum viable BIOS.  So minimal, that the single error message that it can print–“no boot disk available – insert a boot disk and press any key to continue” – neither looks for boot disks nor does anything at all, not even print the message again, when you press any key.  It literally hangs there.  Apple’s position seems to be “if people want a reasonable boot environment, maybe they shouldn’t be emulating floppy disks”.

Apple's representatives could not confirm the authenticity of this statement at press time

Apple’s representatives could not confirm the authenticity of this statement at press time

And Apple shipped the BIOS, and it ran Windows on internal hard disks, and the customers saw it was good.  And it was evening, and it was morning, the fifth day. And thus there is no way to convince your Mac’s BIOS emulation layer — other than by writing your own BIOS emulation firmware — to touch a USB drive.  It won’t happen.

What if my USB disk is “internal”

You may say, “okay.  Boot Camp only works with internal disks.  But what exactly would happen if you install OSX onto an external disk, boot into it, and use Boot Camp on the ‘internal’ (booting) disk?” Checkmate, Apple.

Well, I’ll tell you what will happen.  You’ll launch Boot Camp Assistant, press Continue, and… nothing will happen.  The console will helpfully log that “index 0 is beyond the bounds for empty array.”  Checkmate, Windows user.

And as we’ve talked about, it’s really the firmware on your computer that is the culprit in this story.  The BIOS emulation hiding in your firmware could care less what disk the Boot Camp Assistant is running on.  The BIOS plain won’t poke a USB disk period.

What if we wrote our own special bootloader to boot from USB devices?

There is such a bootloader and it’s called PLoP.

So the first problem is, “how do we install Windows on an external disk so we can boot it?”  Because it turns out that Microsoft’s installer is really, really not on board with external installs.  But through various machinations of wiring up Virtual Box and VMWare Fusion with lots of undocumented options to make external disks look like internal ones, I managed to install both Windows 7 and Windows 8 to an external disk.

And here’s the spoiler: however my Macbook Pro’s USB ports are configured, Plop can’t make heads or tails out of them.  It loads UHCI, OHCI, and EHCI drivers (the difference is entirely political) and none of them have whatever secret sauce it takes to probe a MacbookPro 8,2’s USB subsystem.  Their documentation makes a suspiciously high number of references to DOS drivers, so my best guess is that they are part of the same ecosystem of fail that I have previously written about, and nobody really cares enough to make it work.  But whatever crack they are on, it won’t work for you.

So I’ve heard EFI solves my problems

You may have heard that EFI solves all your “how do I boot [strange operationg system] on [unusual configuration]” problems.  In particular, this is an especially promising line of inquiry because the author has spent a significant amount of time reverse engineering Apple’s EFI implementation.

So a brief history of this blog post.  It started as “How to install Windows 7 on an External Drive with EFI with Parallels.”  Now Windows has never really been on the “fat binary, same bits everywhere” bandwagon that OSX has been on.  The reality of the situation is that subtle things like parameters to command-line tools vary depending on whether you’ve booted Windows in UEFI or BIOS mode.  It goes without saying that running the Windows Installer in UEFI mode creates a UEFI installation, and running the Windows installer in MBR mode creates an MBR installation, and never the two shall meet.  This is problematic because I don’t (and most ordinary people don’t) have access to real UEFI systems that can represent a USB hard disk as an internal disk to the Windows installer.  But unlike the MBR case, the VM story for EFI is pretty shaky.

Parallels doesn’t support EFI, so then the title became “How to install Windows 7 on an External Drive with Virtual Box”.  Virtual Box doesn’t support EFI well, so it became “How to install Windows 7 on an External Drive with EFI with VMWare Fusion”.  But it turns out that EFI support in Windows 7 is more than a little half-baked and actually just hangs at the Apple menu in my testing, so it became “How to install Windows 8 on an External Drive with EFI with VMWare Fusion.”

But our friends in Redmond don’t really understand how to author an ISO that works with EFI, so it became “How to fix some disk image Microsoft gave you so it’s actually bootable  by using arcane tools from Windows versions you’ve never heard of, without actually having any Windows installed.”  (The fix involved, and this is not a joke: downloading a 2GB ISO called the “Windows 7 Automated Installation Kit”, despite the fact that you are neither installing Windows 7, nor doing anything that can be considered “Automated”.  Followed by extracting a 500KB utility called “imagex” from that 2GB ISO.  The 64-bit version is called F3_imagex, the 32-bit version is called F1_imagex, and the F2 is, let’s say, VAX.  Then you booth this monstrosity using the following chain of fail: a network drive emulating a local HFS+ drive containing some Apple file format that is secretly ISO with a different extension that you can only create by emulating certain 2011 Macs, emulating a CD-ROM drive that emulates a USB drive that emulates another CD-ROM drive that emulates a floppy disk.  Which, I can only assume, emulates a poor confused software developer punching holes in cards that emulate butterflies.  You can read about various methods to accomplish this here.)

I’ll save you the trouble and give you the spoiler: the EFI that PCs use and the EFI that Macs implement has diverged.  Just for starters, there’s the U in UEFI.  This would be because Microsoft set up their own little “cross-vendor” standard that the Linux folks are all up in arms about.    The reality is that they’re close enough that some people report it works sometimes, if you delete some drivers from your Windows install.  In my case (MacBookPro 8,2 and also MacMini 5,1) it would start up from an external disk only once out of ten, the other times BSODing (I believe the politically correct term these days is Bug Checking) with IRQL_NOT_LESS_OR_EQUAL which is apparently the worst one to debug, with a little bit of INACCESSIBLE_BOOT_DEVICE just to keep you on your toes.  And rather than emit a Minidump which is supposed to contain debugging information to track down your kernel issues, it would just keel over and die.  And the one time it would boot–by this time Windows has figured out something is wrong at some point during the first nine unsuccessful boots, so it very helpfully tries to drop you into one of various “you should really fix Windows” type modes that are the modern equivalent of whatever we called “safe mode” ten years ago.  Hope you like being in safe mode all the time.

Can’t Apple just implement UEFI 2?

Probably.  The problem is that UEFI just recently stabilized, so a move to UEFI v2 before now wouldn’t have made any sense at all.  And, if it does make sense, it probably only makes sense for new Macs.

You see, Apple’s EFI implementation currently does a lot more than you can possibly imagine, and is really more of a micro OS at this point.  (Fun fact: did you know recent MBPs include not only two random reasonably-powerful microcontrollers, but also an 80MHz ARM CPU?  We’re not really talking “firmware” anymore, this thing is a multi-CPU, multi-architecture OS.)  At the point that your Mac ships with a handful of extra CPUs that you didn’t know about, it’s not a great stretch to expect improved UEFI compatibility in the works for the subsystem.  But I definitely wouldn’t hold my breath on existing models.

Is that all you’ve got?  You really can’t tell me anything more useful to solve my problem?

In fairness, there are some things I didn’t try:

  • I’ve heard good things about EFI compatability on the very very newest Macs, although YMMV.  A good test case is to build the proper UEFI-bootable DVD, take it to an Apple store, hold down alt, and try to boot it.  If it powers up, there’s some remote hope that Windows might reasonably work, although you don’t own such a Mac if you’ve landed on this page, so this is absolutely of no help to you.  Also, modern Macs tend not to contain optical drives, so there’s that.
  • I have heard conflicting reports about to what extent Thunderbolt HDs might work better than USB from a “doesn’t BSOD on startup” perspective.  After all, they are a lot more “PCIish”, so it’s possible that they might look more “internal” from Windows’ POV.

In terms of something that might actually help you, I present the following algorithm:

  1. Visit www.amazon.com
  2. Buy whatever SSD is twice your current size
  3. Receive package from UPS

I can pretty much 100% guarantee that is this is the single viable solution that requires the least amount of new hardware.  Every one of the solutions remaining greatly exceeds the cost of just doubling up on internal storage.  Sorry.

Like this post? Contribute to the coffee fund so I can write more like it.

Comments

  1. db
    Sat 13th Jul 2013 at 10:46 pm

    4 things:
    – Either you are a lot older than I am guessing you are or you spent way too much time learning some very arcane computer history.
    – You may have too much time on your hands.
    – I assume you actually need to boot Windows on the hardware or you would just install it as a VM (with Parallels) then move the VM file over to your external drive.
    – As someone who was around on “the first day” I really enjoyed your recap of the history. I also read and enjoyed and found useful several of your other articles.
    Cheers!
    db

  2. Jenny
    Sat 20th Jul 2013 at 3:40 am

    I understood perhaps half of the words in the article but still found it informative and very entertaining.

    As I have a mac which is refusing to even install windows via bootcamp (white screen of death, yey) I am quite out of options… and apparently this one is another I should cross of the list!

    Good to know though :)

  3. Jenny
    Sat 20th Jul 2013 at 3:43 am

    I’m not sure whether this helps but I just found this (apparently successful) procedure on a forum: In external HD install Mac OS X, and then install windows to same drive via bootcamp.
    Delete Mac OS X partition, Resize Windows partition to size you want.

  4. jmg
    Fri 16th Aug 2013 at 6:11 am

    Thank you very much for this hilarious post. I’ve had 20 very amusing minutes, after spending two hours to find a way to boot a bootcamp installed windows from a formerly internal, now external disk, on my girlfriends computer. … and all that just to submit your taxes

  5. Sat 04th Jan 2014 at 5:12 pm

    Drew, your well-written and highly informative piece was a walk down memory lane for me. In the good old days you just set your front panel switches on your PDP8, loaded your paper tape and away you went. Ah well, times have changed, as have we all.

    FWIW I’ve had some success booting a Mac Pro 4.1 (early 2009, OSX 10.8.4) into W7 SP1 Ult on an external RAID0 array (2xM4 64GB SSD’s) connected to a Marvell 88SE9230-based PCIe X2 4-port SATAIII controller card, Syba’s SI-PEX40057 (cheap-cheap).

    The array was created with Marvell’s MSU app in Win7 booted from an internal bootcamp drive, with the SATA card in slot 4. After a reboot to OSX, Disk Utility was used to partition the RAID0 drive as GUID/HSFJ. iPartition was then used to shrink that partition to a minimum size (214.6 MB for me, YMMV) and partition the rest for MS Data w/NTFS format, “Visible to Windows” and “Active”. Finally a previously-saved W7 SP1 Ult winclone image made from the internal boot camp partition was restored to the MS partition on the external RAID0 disk. Reboot, wait for the chkdsk in Win7 (invisible, just a loooong blank screen, then restart) and voila, bootcamp on an external disk – a RAID0 array, even.

    But not all is well. I have a problem, a problem of greed and stupidity. I have a second Syba card, an SI-PEX40058. Same Marvell chip but 2 SATA + 2 eSATA ports. And I have another RAID0 array I want to use.

    Obviously I’m criminally oblivious to the risk of RAID0 configurations on consumer SSD’s, but I confess, I may have experimented with “drugs” at MIT back in the 70’s, so that’s the least of my flaws.

    Anyway, if I have both cards installed (no matter which slots I pick) the W7 boot process reaches a blank screen and stays there, forever, or at least as long (45 minutes) as I’ve tried waiting.

    It wasn’t always like this. Once I had everything working, if you call a 5+ minute boot delay (blank screen for 2 1/2 min, brief card bios displays, blinking cursor screen for another 2 1/2 min, a weird green Microsoft loading bar and finally login) “working.”

    But, silly me, after numerous efforts (too many to list here) to mitigate this galling delay, I finally took the drastic step of updating the BIOS and firmware on the cards, from a binary image found on “station-drivers.com”, a French website, as Syba and Marvell offer nada. And now darkness lies over the land.

    Syba, sadly, disavows any knowledge of firmware flashing and cannot provide an image of the original firmware – if that is indeed at the root of my current problem. Because it could be any one of so many other things.

    There’s the slot the cards are plugged into: same as before but still a question because only Slot 4 for boot and Slot 2 for the other array worked then. Or the ports that the array’s drives are plugged into: also restored as before but always confusing, as the port numbers don’t match the physical location of the sockets. There’s the partition map on the boot drive array: the firmware flash necessitated recreating the array and the partitioning process described above. And the winclone image, taken from the same drive array when working but shrunk to fit on the infinitesmally smaller partition created after flashing.

    Oh, I could go on but the miracle of factorial growth in the number of possible combinations (compound interest, cry your heart out) makes this all quite academic. I’m stuck at a blank screen.

    Gee though, it was fun while it lasted, if you don’t count that 5+ minute boot delay. And I can still run my Mac Pro in W7 SP1 Ult on an external RAID0 array. But I’m greedy and want another, and I’m too stupid to get it back.

    Now let me be clear about one thing: I understand that those faults of greed and stupidity are my own. They are not the fault of Syba, or Marvell, or the cards’ BIOS and firmware, or of OSX. They’re not even Windows’ fault, and think how rare that is.

    For that tiny tithe of understanding I am grateful. And if you, Drew, or any other readers can help further, I’d be ever so much more so.

Add comment

Copyright © 2011 Drew Crawford, All Rights Reserved
Powered by WordPress

Page optimized by WP Minify WordPress Plugin