07 July 2008

XP Repair Install

Technorati tags: ,

Re-installing Windows XP isn't a good idea as a blind first step in troubleshooting problems, but there are specific contexts where it is necessary, as the cleanest way to "make things work".  One of these contexts is after a motherboard change that invalidates XP's core assumptions, typically causing a STOP BSoD on any sort of attempted XP boot (from Safe Cmd to normal GUI).

This is the situation that edgecrusher is in, as posted in comments to the previous post in this blog, and this post is my response.

Before you start

Firstly, I'm going to assume you have all the necessary installation and drivers disks, have your XP product key or retrieved this via Nirsoft Produkey or similar, excluded malware, and verified RAM overnight e.g. via MemTest86 or MemTest86+ and hard drive e.g. via HD Tune

Make sure the edition (OEM vs. retail, Home vs. Pro, etc.) of the XP installation disk you will use for the repair install is one that matches your product key, that the disk actually has the ability to do a non-destructive install (as many OEM disks do not), and that the disk can be read without errors (as tested by copying all files to a subdir on the hard drive before you start).

It's a good idea to make a partition image backup of your XP installation before you start, using something like BING.  Simply copying off every file is not enough, because unlike Windows 9x, XP will not work when copied in this way.

Also before you start, you may want to uninstall any OS-bundled subsystems that you've upgraded past the baseline of your XP installation disk, such as IE7 or recent versions of Windows Media Player.  Things are cleaner and more likely to be "supported" if you uninstall these before the repair, and re-install them afterwards, plus you'll have valid entries in Add/Remove Programs should you need to uninstall them again later (e.g. as a troubleshooting step).

Several sites describe the XP repair install process, starting from how to start the process, and going on to a step-by-step slide show or providing more detail.  In this post, I will mention a few specific gotchas to avoid...

137G capacity limit

If your hard drive is over 137G in size, then the Service Pack level of the Windows XP installation disk must be at least SP1 to install, and SP2 to live with.  In other words, you cannot safely install XP "Gold" (SP0) on a hard drive over 137G, and should apply SP2 or SP3 over an XP SP1 installation. 

If your install disk pre-dates SP1, you need to slipstream a later Service Pack into this and make a new installation disk that includes SP1 or later, built in.  Your other option is to install XP "Gold" onto a hard drive smaller than 137G, apply SP1 or later, and then use a partition transfer utility to copy the partition to the larger hard drive where the partition can then be resized to taste.

XP "Gold" has no awareness of hard drives over 137G and is very likely to mess them up.  XP SP1 is supposed to be safe on such hard drives, but there are some contexts where the code that writes to disk is unsafe and may cause corruption and data loss; from memory, these contexts typically apply to C:, e.g. writing crash dumps to the page file.  XP SP2 and SP3 are truly safe over 137G.

F6 driver diskette

Yep, you read right; that's "diskette" as in "ancient crusty old stiffy drive"! 

Most current motherboards have S-ATA hard drive interfaces that are not "seen" by the native XP code set (affecting Bart and Recovery Console boot disks as well).

The trouble is, the latest PCs often have no diskette drive, and the latest motherboards often have no legacy diskette controller.  You may come right with an external diskette drive plugged in via USB.  You'll also have to find and download the relevant driver diskette image and make a diskette from this, if yours is missing or unreliable.

If you use a USB keyboard, and this is not initiated at the BIOS level, then your F6 keystroke to read the driver diskette will be missed.  If so, you can plug in a PS/2 keyboard... as long as your new motherboard has PS/2 sockets; the newest ones don't.

Sometimes your mileage may vary, depending on the mode that your S-ATA is set to operate in CMOS Setup.  RAID and AHCI will generally not be "seen" natively by XP's code, whereas IDE mode may be.  But some nice S-ATA features may not work in IDE mode, e.g. hot-swapping external S-ATA or NLQ, and changing this after XP is installed may precipitate the same crisis as the motherboard swap... requiring a repair install to fix, again.

All of this is a reason why I consider the XP era to be over, when it comes to new PCs.  I appreciate how old OSs run beautifully fast on new hardware, and how attractive that is for gamers in particular - but XP's getting painful to install and maintain, and this is going to get worse.

Duplicate user accounts

Later in the GUI part of the installation process, you will be prompted to create new user accounts.  You can try to skip this step (best, if that works... I can't remember if it does), or create a new account with a different name that you'd generally delete later. 

But many users are likely to create a new account with the same one as their existing account, and that's likely to hurt... 

The two accounts will show the same name at the Welcome screen, but both will be selectable via this UI; I have no idea what will happen if you were to force the more secure legacy logon UI, which requires the account name to be typed in.

Each account will have a unique Security Identifier (SID), which is the real "name" used behind the scenes - but you can't login with that.  There will also be separate account subtrees in "Documents and Settings"; the one with the plainest name is likely to be the original, and the one with numbers or the PC name added to it is likely to be for the newly-spawned account.

At this point I'll mention another user account hassle that I generally don't see, because I avoid NTFS where I can.  If you find you can "see" your old user account's data, but aren't permitted to access the files, then you may have to "take ownership" of these files from a user account that has full administrative rights. 

This issue is well documented elsewhere; search and ye will find!

Broken update services

It's a given that the "repair" is going to blow away all patches subsequent to the baseline SP level of the XP installation disk you are using, unless you've slipstreamed these into your installation disk.

What's less obvious is that after you do the "repair" install, you won't be able to install updates.  It doesn't matter whether you try via Automatic Update, Windows Update or Microsoft Update, the results will be the same; the stuff downloads OK (costing you bandwidth) but will not install, whether you are prompted to restart or not.

The cause is a mismatch between the "old" update code within the installation CD, and the newer update code that was controversially pushed via update itself.  I can see Microsoft's logic here; if you ever wanted updates to work (e.g. you'd chosen "download but don't install", or disabled updates while planning to enable them later), then the update mechanism has to be updated - but doing so, invalidates the original installation disk's update code.

This topic is well-covered, as is the fix; manually re-registering a number of .DLLs that are needed for the update process to work.

Broken settings

It's often asserted that a repair install "won't lose your settings", and is yet waved around as a generic fix for undiagnosed problems.  Part of why it sometimes works as a "generic fix" is precisely because it can and does flatten some settings, which may have been deranged to the point that the OS couldn't boot!

So if you do apply any non-default settings, you should check these to see if they've survived.  I always check the following, and can't remember with certainty which ones survive and which don't:

  • System Restore (may be re-enabled on all volumes)
  • System Restore per-volume capacity limits
  • Automatically restart on system errors
  • RPC Restart the computer on failures (may survive)
  • Show all files, extensions, full paths, etc. (may survive)
  • NoDriveTypeAutoRun and NoDriveAutoRun
  • Standard services you may have disabled
  • Hidden admin shares, if you'd disabled them
  • Recovery Console enabling settings
  • AutoChk parameters in BootExecute setting
  • Shell folder paths
  • Windows Scripting Host, if you'd disabled it
  • Settings detail in IE, including grotesquely huge web cache
  • Windows Firewall settings; may be disabled if < SP2 !!
  • Anything else you've dared to change from duhfaults

It's particularly crucial to enable the Windows Firewall (or install a 3rd-party alternative) before letting your PC anywhere near any sort of networking, especially the Internet, if your installation is "Gold" or SP1.  Not only do these dozeballs duhfault to "no firewall", they're also unpatched against RPC (Lovesan et al) and LSASS (Sasser et al) attacks, so you'd be "open and revolving".

By now, the original PoC Lovesan and Sasser worms may be extinct, but these exploits are often crafted into subsequent workaday bots and worms.  You may still get hit within an hour of plugging in the network cable if so, and probably before you can pull down updates for the OS, antivirus scanners, etc.

19 comments:

AlexFielder said...

cheers for the really useful information (links, things to do before you start etc.) in this post - I'll certainly have your page open on my other machine before attempting anything major on my new-build system.
To save any BSOD shenanigans, I may just flatten the whole lot and start again - I do have a slipstreamed Windows XP Pro SP2 disk I can use, although it probably won't contain the correct drivers for the new motherboard. I guess I should make a new disk using nLite and include the drivers that came with the new kit.

I'll post again once the system is up and running. :)

Cheers,

Edge.

Anonymous said...

"All of this is a reason why I consider the XP era to be over, when it comes to new PCs. I appreciate how old OSs run beautifully fast on new hardware, and how attractive that is for gamers in particular - but XP's getting painful to install and maintain, and this is going to get worse." From Chris Quirke's post

Chris, are we the only ones who love all the legacy stuff? Your 9x web page has been invaluable to me and my work. My modern machine is specifically all legacy so it can support 98 Second Edition on a different hard drive than XP Professional. Heck, I have to go to great lengths to still have a somewhat modern machine that stretches 98 Second Edition to its limits and beyond. It has gotten so bad that despite having all the proper ram configurations that I still must remove a ram module to have 98 SE to run.

Heck, I still have my old IBM PCjr in storage in NYC where I grew up in the 70's and beyond. My folks rent out their place in NYC because it is way too expensive for us to live there anymore. In addition, I love my IBM machine at my folk's place in New Mexico that is an IBM 486 with Windows 3.1 that is okay but I really like the DOS 5.02 that came with the machine. I feel so comfortable in a text interface and do not have to deal with the junk and slowness of getting things done in a GUI interface. I much preferred having my machine as a stand alone machine and do my work at home and job work on the job and so now all job work as far as VPN access and work intranet access will stay off my PC unless forced to do otherwise.

You already know my story before of the hacking of my old machine via the piggybacking and hacking of the company's APS network. I will now only go into details with you in private if you want more information about that scenario. Also, I just seem to attract weird software vulnerabilities at work that no one else has any clue about. Again, I will go into detail with you in private. I am very thankful for good guys like you that help people like me to get their work done.

It seems like modern IT People know some stuff but are completely lost as far as 9x is concerned which is a shame. Microsoft did their job very effectively of now having one source code line and we are all suffering the consequences. Am I alone in my thoughts on these points?

Chris Quirke said...

Hi edgecrusher!

You won't see BSoDs if you initiate the repair install from CD boot, because the installed OS will never smell the new hardware without having the appropriate boot code in place.

So I'd do that, rather than a destructive rebuild, or an attempt to boot the old installation on the new hardware to initiate the repair install from within Windows.

Drives not needed for boot, can be added later. Drivers needed for boot have to be done at install time via the F6 diskette prompt.

Before you install on the new motherboard, check the following in CMOS Setup:

- CD boots before HD
- correct S-ATA mode selected

S-ATA may be set as IDE (or "legacy"), AHCI (or "enhanced") or RAID. Unless RAID, you prolly want AHCI for the rest S-ATA performance, but that will almost certainly need an appropriate F6 diskette for boot drivers.

I don't think doing a clean install of XP will solve any problems you might have with S-ATA mode, F6 diskette, etc. i.e. if for some reason you can't get drivers in place F6, you are just as hosed either way.

Chris Quirke said...

Hi Dan!

Hmm... "deep legacy"... not really a labour of love if you have to do it for a living :-)

For XP on new PCs, I'd consider shelling this within a full-performance virtual machine, either on Vista or Linux. That may seem extreme now, but in won't in a year or few's time, when >3G RAM and 54-bit OSs are the norm.

For Win9x, you do have to curb processor speed and keep HD visibility < 137G. So full-performance virtualization may miss the spot, and you may want a deeper level of emulation, as per DOS.

For DOS, I'd use the DOS mode from Win98xx with MSDOS.SYS set yp BootGUI=0, rather than an older stand-alone MS-DOS. For most DOS apps, this gets more conventional memory, and thus works better.

But the speed and capacity norms of DOS are such that I'd rather emulate it as an alien processor (as you'd do for old ROM-based coin-op games, ZX Spectrum, etc.) so that CPU timings can be emulated in real time - not just for speed, but also for consistency across different instructions.

As to IT folks knowing 9x; it's quite possible not to, given that XP came in 6 years ago - you could do a BSc in that time.

More to the point, at the time Vista came out, it was 5 years ago or so since the last time hardware and software vendors had to endure a major OS change. That may explain partly why they've been so useless at it :-)

Anonymous said...

Chris, do you mean 64 bit instead of 54 bit operating systems? In addition, what does you could do a BSc in that time mean?

Chris Quirke said...

Yes, 64-bit not 54-bit (typo).

A BSc is what, 4 years? So you could start that degree when XP was a newish OS, and graduate while it's still the current OS. Other shorter training, even more so; XP may well be the only Windows you'd experienced in any technical depth.

Next, consider programmers in a large company. You arrive as a drudge-drone, doing the more boring and perfunctionary code, and work your way up - soon, you aren't a front-line coder anymore, but maybe a project manager or something.

So at the time that Vista came out, there may be no coders familiar with the process of adapting to a new OS version. Even your immediate team leader may not have lived through that; it could be your company didn't even exist in those days.

I'm sure that by now, folks have moved around inside Microsoft that the original Win9x team will be scattered far and wide, and those who actually did the coding, may be coding no more.

Anonymous said...

I think you mean Bachelor of Science degree. You make some good points in your reply. So will the 9x source code be locked away as Microsoft's IP until it has been so long that it is useless. This seems really selfish to me when Microsoft could enhance everyone's security by selling it to another company or government to enhance the information security and safety of the world's computers. That is why I have just started exploring Ubuntu Linux.

There was an interesting article on the front page of The New York Times today (July 10, 2008 in the States) about China and its spying on the United State's computer network(s). You should read it if you get the chance. Anyway, my friend Will mentioned how countries could also mask their hacking attempts by using servers through other countries to make it appear that they were the ones attempting the port scans and other cyber attacks. I remember a few years back hooking up the last version of Zone Alarm Professional for 98 Second Edition without a router at my folk's house in the States and seeing where the attacks originated. The most dangerous port scan attempts appeared to originate from China with some also from the U.S. itself and some from Russia. Just Food for Thought.

Chris Quirke said...

Hi Dan - yes, that's the degree I mean.

I'm in two minds about abandonware, and whether it should be released into the public domain at the source code level.

I do think that's true for a lot of stuff, such as music, medicine patents, etc.

There is an added aspect to software that may be abandoned as a product, but that shares code, or even code concepts, with products that are still in use.

I think one may expect a vendor to disregard re-use of this old stuff in object code form. But I don't think one can expect them to release the source code, or relinquish the rights to develop from this.

Anonymous said...

What about if it would help the world's safety of the Internet infastructure against the hackers who are so intent on wrecking havoc on cyberspace?

Chris Quirke said...

Dan, I don't think the Win9x code has any positive relevance to today's DNS problems.

Think about it; if Win9x has any safety strengths (over NT) for consumers, it's because it is not designed as a network OS - so it waves less surface to the Internet.

In contrast, a DNS does little other that wave surfaces to the Internet - that's what it has to do.

Face it; Win9x's time has passed. Not only will it not run on modern systems (>137G hard drives, processor that are "too fast", heavy reliance on USB, post-AGP chipsets), it no longer has safe edge-facing software available for it - Firefox no longer runs on Win9x, and Avast is the last free av that will run on it.

Anonymous said...

Chris, Firefox 2.0.0.16 runs on Windows 98 Second Edition which I am now using as my main operating system and it is current and supported to the end of 2008. Mozilla Firefox 3 does not run on Windows 98 Second Edition of course but Mozilla Firefox 2.x is still supported and I still use the legacy Mozilla 2.x on Windows XP Professional.

I am the only true old-school die hard left I guess and I posted the 98 error I got when the Albuquerque Public School Domain was hacked and I lost XP Professional but all the hackers could do with 98 Second Edition was a denial of service error.

Please see secunia.com and research the vulnerabilities in 98 Second Edition, XP Home/Professional, Windows 2000 and Windows Vista as well as Internet Explorer and Mozilla Firefox. Heck, I even recently hooked up my IBM Keyboard from my 486 PC to my dual-boot 98 Second Edition and XP Professional because I want technology that runs well and I like the click of the IBM Keyboard. Please correct your information and have a great day.

Chris Quirke said...

Hi Dan!

I was thinking in terms of current versions, looking forward, as one would have to do if doing new deployments of Win9x.

Yes, the old Firefox 2.x is still being patched, and works in Win98+, but 3.x doesn't. I don't know about Opera or (shudder) Safari,

On av, Avast is the last free resident av that still works on Win9x, and when I set it up on a Win98SE PC recently, the PC no longer shut down properly.

I have a hunch that if I walked in to Avast and banged my fist on the table about this, they'd be more likely to drop Win9x support than troubleshoot this issue for a revision of thier product.

The point about all this is that although the OS is old and niche, therefore unlikely to be attacked, the edge-facing surfaces it has to use are not - and attacks have moved from OS to common "edge" software, e.g. Acrobat Reader, QuickTime, web browsers etc.

When you put that into the picture, it starts to erode the safety case for using Win9x boht now, and especially in the years to come.

Alex Garfield said...

True, I see your point. Chris, I am happily posting in Windows 98 Second Edition right now having downgraded from 2 gigabytes to 512 megabytes because I am so old school. Anyway, the only attacks I have encountered so far have been denial of service attacks and that has happened with Internet Explorer. I have not run into any other attacks on me yet. I use Comcast Cable connection and Comcast was one of the first to patch their servers from DNS Pollution according to information from us-cert.gov, an agency of the Department of Homeland Security (DHS) here in the U.S.A. BTW, the feds being DOD and DHS are now finally really interested in my research and want all my data about the hacking of the APS Network so their DOD Lab can try to figure out exactly what happened and soon the lawyers will be involved to represent me. shudder lawyers -- grin

Chris Quirke said...

Heh - sounds a bit "deep legacy". If I want to play the game I wrote in PICK R83 back in the late '80s, I have to find a PC with < 16M RAM and a working 360k diskette drive :-)

Anonymous said...

Some insights about how the repair setup decides when to "detect" your windows installation:
(or: windows setup repair internals, under the hood etc...)

From my tests, the following conditions (files) must be satisfied:

1) Boot stuff:
C:\NTDETECT.COM, C:\NTLDR
BOOT.INI must include your windir directory.

2) Kernel & Executive:
%windir%\system32\ntdll.dll
%windir%\system32\ntoskrnl.exe

3) Registry:
%windir%\system32\config\software
%windir%\system32\config\system


4) Drivers (can be empty dir but must exist):
%windir%\system32\drivers


My guess is that the setup will check your kernel & executive (ntoskrnl.exe, ntdll.dll) versus the disc's ones, and only if they match it will detect the installation and allow repair.
This is only a guess i havent tested it yet.

Chris Quirke said...

Thanks, Liran!

I've often wondered about what constitutes an "installation", but haven't tested it in the NT family... in Win9x for example, it would usually be the presence of Win.com (rename that away, and OEM Win9x will "upgrade" the existing installation just fine).

There are three states that are interesting:

1) The minimum that can be treated as an installation

2) The maximum that can be treated as *not* an installation

3) What constitutes an "installation in progress"

I've only tested (3), and can't remember the results now {g} but I needed to know as this state derails Bart boot and access.

How did you get bold in comments? Bwoo! {bows in respect}

igeorge said...

I have a computer whic initialy booted from C: . Then it crashed and i installed Win XP pro on D:. Later i added a Raid 1 and installed on it XP pro again- the letter is H. Under computer, manage, disk management i have c: = Boot, D = System and H is just healty with no status. The boot.ini and ntldr are on drive D only.
I WISH that i can make H the boot , because it is a raid 1 , then later on i will remove the disk 1 which is C and D.
Boot.ini:
[boot loader]
timeout=30
default=multi(0)disk(0)rdisk(1)partition(1)\WINDOWS
[operating systems]
multi(0)disk(0)rdisk(1)partition(1)\WINDOWS="Microsoft Windows XP Professional Drive H" /noexecute=optin /fastdetect
multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP Professional Drive D" /noexecute=optin /fastdetect
C:\CMDCONS\BOOTSECT.DAT="Microsoft Windows Recovery Console" /cmdcons

Can you help please ?
Thank you
Ion George .

Chris Quirke said...

Hi, Ion!

You have a few issues, such as the 137G limit (OK if your XP is SP1 or later and/or hard drives are smaller than 137G) and the probable need for an "F6 driver diskette" if you want your RAID hard drive to be bootable.

If those two issues are OK, then you could remove the non-RAID hard drive so only the "H:" is available, boot your XP CD, and do a repair install onto the RAID hard drives.

RAID 1 implies two (matching?) hard drives operating as one of the same capacity, i.e. sacrificing the additive capacity of two separate hard drives for the reliability of redundancy, writing the same data to both drives so all is well even if one dies completely.

Is that what you have? I ask, because you speak of 2 rather than 3 hard drives in total.

Anyway, if you do the repair install with just the RAID 1 in place, and have bootability assured (F6 diskette, 137G safety) then you also have to ensure the RAID boots before the loose hard drive you put back, as determined by CMOS setup.

If that isn't possible, you may have a problem even if you make the added loose hard drive unbootable (e.g. by ensuring only logical volumes on an extended partition, so no primary partitions that can boot), if the Boot.ini partition syntax is incorrect - though you can fix that via Recovery Console /FixBoot or manual edits from a Bart CDR boot.

Chris Quirke said...

Ah, Ion; on re-reading your post, I realize my earlier reply wouldn't work unless you'd have first copied the contents of D: (i.e. the OS files) to RAID 1 H:, and then removed the non-RAID hard drive (C: and D:) before attempting the repair install.

You'd have to ensure...

1) That the RAID is bootable, at the BIOS level - not always the case if the RAID controller is an add-on card

2) That the OS is safe over 137G and/or the RAID 1 hard drive is smaller than 137G

3) That you have a suitable F6 driver diskette, so that the repair installation can build in the appropriate driver code for NTLDR to boot the RAID hard drive

It's (3) that is most likely to bite you. I'm sure you're already running XP SP3 or at least SP2, so would be safe for 137G, but you may be relying on installing from an older XP OS CD and then applying the latest SP after the installation - that won't work if the hard drive is over 137G and the OS CD is "Gold". You'd have to slipstream the SP, and the easiest way to do that is via the free nLite utility.