Chris Quirke's Blog

C-Net’s Downloader Pollutes “My Documents”

2011-11-20T07:31:00.001-08:00

You may have noticed C-Net have started using a downloader stub when you download software from them. The stub adds no value that I can see (it claims to be “more secure”) and tries to push unwanted bycatch, such as browser toolbars etc.

So far, so normal and nasty, but there’s something else that makes this totally unacceptable IMO – it changes the location where your download is saved, ignoring your browser’s settings and providing no UI to see where this is beforehand, or change it.

And where does it save the download? The Windows Vista/7 Downloads shell folder? No; in “My Documents”. So now you have infectable incoming code dumped into your “data” set, where it will pollute your data backups too.

I was wondering why I was seeing so much Incredimail, Babylon Toolbar and other junk on client systems – now I know why. What I now know, is that I must also clear these code downloads from the data set.

SkipRearm Setting for SysPrep Failure

2011-10-04T07:58:00.001-07:00

Technorati Tags: SysPrep,Windows

Here’s how it goes; you have an un-activated Vista or Windows 7 reference system ready for SysPrep and .WIM harvesting, but SysPrep fails. You search, and find articles that mutter about adding a “SkipRearm” setting to an “answer file”, but get stuck there if you don’t know how to apply an answer file.

Fortunately, there’s a simpler fix that I found and tested for Windows 7, and it works. For Vista (which I didn’t test)…

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurentVersion\SL\SkipRearm = 1

…and for Windows 7 (as tested OK):

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WindowsNT\CurrentVersion\SoftwareProtectionPlatform\SkipRearm = 1

The nice thing is, you don’t have to fiddle with “answer files”, or .WIM mounting and manipulation in WAIK.

This fits with the simplistic way I use .WIM imaging; I use only WinPE 3.0, ImageX, and the GimageX GUI wrapper for convenience. My WinPE 3.0 is standard other than the addition of GimageX and ImageX, and a setting to prevent the WinPE boot from falling through to boot the hard drive if no key is pressed (sorry, no link for that).

When building a system, I partition via BING, format the prospective C: to NTFS via WinPE, then apply the .WIM, so I have a baseline installation that when booted, will resume Windows Setup as part of what SysPrep did prior to the creation of the .WIM image. I do the first boot OFFline, and kill the duhfault setting to automatically activate Windows.

Then I update and install free software to taste, until the new PC is generically fully set up. I use BING to image the C: partition for safekeeping (in case SysPrep screws up), then run SysPrep and Generalize the new PC. I then boot WinPE to capture C: as a new and updated .WIM, then I boot BING to restore the partition to the state before SysPrep was run. At this point I can apply client-specific changes, activate Windows, and ship the new PC.

SysPrep does not maintain undoability, and tends to screw up. When it does, you can be left with no bootable reference system and no usable new .WIM, so I again stress the need to image-backup C: before SysPrep. If you’ve done that, you may prefer to restore that image rather than wade through and clean up after SysPrep’s effects.

Key safety

One of the things you want to avoid when working with what you hope to harvest as a reference .WIM, is inadvertently activating the build, especially with the wrong product key:

Disable the “automatically activate” setting
Keep new PC offline from build until first backup image of C:
Re-check “automatically activate” setting before going online
Do the “image backup, SysPrep, restore C:” sandwich
Check the current key before activating
Activate before shipping as new PC

When I tested SysPrep with SkipRearm, I did not enter a product key when prompted, and used Nirsoft’s Produkey tool to check the key. This showed a key other than that of the client, so SysPrep had stripped that OK, and presumably fallen back to some previous or fake key. When I restored the pre-SysPrep BING partition image as C:, this showed the expected client’s key, as I’d entered when originally starting the build from the previous .WIM

Final tip; if/as SkipRearm doesn’t reset the full grace period for activation, you may want to minimize the days spent between restoring the previous .WIM and capturing the next one.

Mint/Linux, Sandy Bridge, Blank-screen Intel Graphics

2011-10-01T17:01:00.001-07:00

Mint 11.4 64-bit installs on Cold Lake H67 Sandy Bridge motherboard PC OK, hard drive boots to grub2 OK, but Mint boots to a black screen with no mouse pointer. The OS is running OK, you just can’t see anything – or almost; a close look at the screen shows a pixel-flickering red line down the left edge and a solid short horizontal white line top left, when a 17” CRT is used.

The problem appears with LCD screens also, and is variable; some boots may be OK. At present, all 3 of 3 new PC builds have done this on first Mint hard drive boot.

The fix: Plug another monitor into the other graphics socket. The desktop will immediately appear on both screens; you can then unplug the second screen and it will work OK (at least for that booted session).

Cold Lake motherboards use the H67 chipset, which in turn interfaces the Intel GPU built into the processor, to DVI and HDMI dual display outputs. Typically I use a DVI to VGA adapter to take the DVI’s analog signal to 17” CRT or more modern LCD, leaving the HDMI unplugged, though from memory I recall similar mileage when a digital-signal LCD was plugged into DVI, or HDMI via HDMI to DVI “pigtail” adapter.

It seems like Linux can’t figure out what display signal to use?

If previous Ubuntu mileage is anything to go by, the “fix” is prolly to wait until the next OS release in the hopes it will use a newer Linux kernel that has a clue about “new” hardware. Bah! Still, that should be due later this month, so let’s see how it goes.

Robot Drivers and Driving Tests

2010-10-11T23:50:00.001-07:00

If we require training and licensing of humans to fly aircraft and cars, then how do those standards apply to software that pilots these things for us?

A key is situational awareness. You wouldn't easily give a blind pilot a flying license, yet effectively that is what Airbus tried to do with an engine control system that "landed" a test plane in a forest. It's one thing having a robot hospital cart that negotiates around people's ankles at walking speed, quite another to do that at road speeds, or while attempting to keep an airliner within its flight envelope.

Perhaps the human pilot or driver is expected to remain in control, on standby and ready to override the robotics? Good luck with that, as attention wanders and distractions take the foreground in the human's mind.

I see Google's already had cars driven by software logic on public roads. I wonder what the traffic cops would have to say about that?

Driver Cure or Driver Curse?

2010-09-07T15:33:00.001-07:00

If you'd just dropped into the PC world last week, you'd think all software was perishable and had to be continuously refreshed. Must always have the latest version BIOS, drivers, etc.

This attitude runs counter to an older wisdom, that the first question when something goes wrong, is: "What changed?" With this in mind, the last thing you want is vendor-driven changes to your code base; in fact, for a critical working system, you want no changes at all.

The logic behind all this is contradictory...

Software vendors make mistakes, requiring software repairs ("patches" or "updates")

This happens so often, you may not be able to keep up with the constant flow of updates

So it's best to let the software vendor push updates whenever they see fit

This boils down to: Trust software vendors to push changes into your code, because they fail that trust so often you can't keep up with the pace of quality repair required.

So, should you always patch, or never patch? Or sometimes patch? If "sometimes", then on what basis do you use to decide what needs patching?

Balancing risks

Some code is so critical, you may consider it too risky to change, e.g. BIOS and device firmware, device drivers, core OS code, and code that is running all the time and can crash the PC if it goes wrong.

Some code is so exposed to arbitrary unsolicited material, you may consider it too risky to leave unpatched, for fear that malware may exploit defects in the code to attack your PC.

Code should never fall into both of the above categories; if it does, you're probably looking at really bad software design. For example, integrating a web browser so deeply into the system that it's indivisible from the system's own UI, would be a bad design decision. Or consider a service so critical to the system's internal functioning that the OS shuts down the whole PC every time the service fails, that is waved at the Internet on the basis it's "networking"; that would be a really bad decision (Lovesan vs. RPC, remember?).

Trust me, I'm a software vendor

The two reasons not to trust a software vendor are incompitence and perfidity. A vendor who claims you "must" leave your system open to a constant stream of fixes, has declared themselves incapable of writing code that can be trusted to work properly.

And frankly, when even "legit" vendors hide deliberately user-hostile code within their products, set to automatically deny you service if its logic considers your license state is invalid (product activation) or distribute rootkits within "audio CDs" (Sony), I'd not trust any vendor's ethics.

Finally, even if you trust the vendor's ethics, you have to look at the mechanics of code distribution. Fakeware abounds, so when a third party claims to serve you fresh code from the vendors you trust, you have to ask yourself how trustworthy is that third party?

You also have to ask why you'd trust a particular software package. Open source advocates would say it's because you can read the source code yourself, or at least feel safer in that others have done this on your behalf. Closed source advocates would say it is unrealistic to read source code yourself, and instead would point to pre-deployment testing that would pick up unwanted behavior before the code was used in the real world.

Patches and updates change both of these equations, because now the code you read and/or tested, is no longer the actual code that is running. Any patch may add unwanted behaviors that favor whoever pushed the patch into your system. For the same reason, you should avoid software that stores "your" settings on the server side rather than on your PC (e.g. Real Player, many Instant Messaging apps) and "Privacy Policies" and End User License "Agreements" that state "these terms can be changed whenever we see fit", as so many do.

The race to patch

There's a race between freshly-released malware, vs. your antivirus scanner that protects your system. When a new malware is found, the antivirus vendor analyses the code to work out how to detect it, then how to safely remove the code, then that logic is packages as an update that your PC's scanner pulls to update your protection.

Compares this to what happens when a new vulnerability is patched. The malware coders can compare pre- and post-patched code to isolate the fix, then work out what the unfixed code did wrong, and thus how to attack that code. The exploit code is then packaged into malware prepared earlier, such as a downloader stub, and that's pushed into the wild.

Notice the similarities between these processes, i.e. recognizing and removing malware compared to extracting and exploiting code defects from studying patches?

If you rely on resident antivirus to protect you, then you are betting on the av vendor to beat the malware in the race. By the same token, you may expect malware coders to be fast enough to exploit your edge-facing code before the patch arrives to fix the defect. Hence the manic rush to patch, for fear of prompt exploit.

It's actually a bit worse than this, for two reasons. Firstly, sometimes it's the malware folks who find and exploit defects before the code vendor learns about these and fixes them. Secondly, software vendors have to ensure their patches don't break any systems, whereas a malware coder just wants it work enough of the time to spread, and doesn't care if it breaks other systems in the process. Less rigorous testing means "faster to market", right?

Self-spreading malware can also spread faster from more systems, and thus beat the patching or updating processes to the punch. Malware can be delivered in real-time via pure network worms, or links to servers that are themselves updated in real time. Often the malware that enters the system is just a downloader stub; it only has to last long enough to pull down the "real" malware, which can replace itself in real time as well.

Edge-facing software

With all this in mind, you can see why one would want to patch edge-facing software as soon as possible. Examples include web browsers, Java, Acrobat Reader, Flash and media players, and anything that is constantly exposed to the outside world, such as software that waits for instant messages or "phone" calls.

The best solution is to remove that edge-facing software, and thus the need to patch it. Do this whenever you don't need that software, when the software or its vendor are too flaky to trust, or when the update process itself is something you want to avoid.

For example, you may catch a vendor trying to shove new edge-facing software as "updates", even when that software is not present and therefore doesn't require patching. That's how Apple used to push Safari to PCs running iTunes or QuickTime, until they were pressurized to stop.

For another example, a vendor may decide you don't need to be asked before updates are pushed, or even told when this has happened. And when you look at that vendor's updater, you find it running as multiple scheduled tasks; then when you look at the details, you find a task that appears to be run once a day, is actually repeatedly run every hour throughout the day. That's the equation with Google, and why I would avoid any edge-facing Google software.

If you can't avoid edge-facing software, then you can protect yourself in two ways; by updating it as soon as possible, and/or by choosing such obscure, small-market-share products that they aren't likely to be attacked. The latter is like living in an unlocked shack in the countryside; that works not because shacks are "so secure", but because there are so few attackers around.

Driver Cure or Driver Curse?

So now we come to Driver Cure, which is a third party product that pulls in the latest versions of your device drivers. Would you want this? I'd say no, for two reasons.

Firstly, device drivers are code that runs so "deep" in the system, that any mistakes are very likely to crash the entire OS, leaving the file system corrupted, data files unsaved, etc. Device drivers usually run all the time, so bad code may prevent the system from being able to boot or run at all, even in Safe Mode. So I definitely don't want unexpected changes to this code, any of which may cause the system to stop working.

Secondly, device drivers are not edge-facing, so the risk of explosure exploit should not be high. That means less reason to patch in haste.

Thirdly, if malware were to be integrated into the system as deeply as a device driver, it would have considerable power and be very hard to remove. So we'd want to know a lot more about third party software that inserts "drivers" into the system.

The "Driver Cure" folks also push XoftSpy, which was one of several hundred fake anti-spyware scanners, until they supposedly "went legit". As such, sites and blogs may no longer call XoftSpy "malware" for fear of being sued; we may instead consider it as a legit antispyware that isn't very good at what it does, and costs money where better products are free.

So, in spite of "reviews" like these, I would avoid Driver Cure and anything else from that particular software vendor or distributor.

Zoundry Raven Double-Spaces On Blogger

2010-06-11T10:54:00.001-07:00

I've started using Zoundry Raven as my off-line blog editor instead of Windows Live Writer, as the latter depends on Live Passport before it can do anything, anywhere, and for some reason that's gone haywire in my case.

It posts OK to my WordPress blog, but on Blogger (here), I see my text paragraphs are separated by two blank lines, rather than one.

Well, HTML always seems to have trouble with white space control; either too many blank lines, or not as many as you'd expect; reminds me of DOS vs. UNIX CR/LF issues in the days of dot matrix printers. Looking at the source code, I see previous posts have the (/p)(p) sequences I'd expect (where I've used parantheses rather than angle brackets to show the codes as text), but the new post has a (/p)(br /)(p) sequence instead.

OK, so I'll clean up my posts in the HTML code window before posting, right? Nope; this is "XHTML" in Raven, and it innocently shows the expected (/p)(p) sequences. So perhaps the transport process and/or Blogger's server side is screwing up - bleh.

AVG 9 Update Blocked By Ashampoo Firewall

2010-06-11T10:35:00.001-07:00

Geek summary: AVGUpd.exe can't pass Ashampoo Firewall when launched within AVG, but works when run explicitly; as a workaround, create a Task to run on startup and every few hours

Firewalls sometimes block antivirus updates, even when all of the antivirus executables have been permitted full Internet access through the firewall. In this case, I hit the problem on a Windows 2000 PC, on which I'd installed AVG 9.0 Free antivirus and Ashampoo Free Firewall.

I searched about this and found several threads at AVG's forums, which usually pointed to an article that lists the AVG executables that should be allowed through the firewall. Other posts elsewhere simply declared Ashampoo as incompatible with AVG.

Even listing every AVG executable in Ashampoo's rules does not fix the issue, which is more subtle than it looks. Specifically, the AVGUpd.exe update tool does work if launched explicitly, but if it is invoked through AVG's general UI, the update fails.

If you run AVGUpd.exe directly, nothing visible happens unless you already have the general AVG dialog open at the time. If you do, then you will see that dialog indicate the update is now in progress, and it works.

The problem may be Ashampoo blocking interprocess communication within AVG (though I had disabled that sort of internal functionality), or failing to recognise when AVGUpd.exe is invoked from AVG's code (perhaps it calls it as a .DLL?).

A fix (aside from ditching Ashampoo, which I'm reluctant to use for the next Windows 2000 PC that needs a free firewall) is to create Start Menu and desktop shortcuts to run AVGUpd.exe, and/or set a Task to launch this when starting the system and/or every few hours during the day. Both approaches appear to work so far.

Sysprep Fails, WinPE Sees Wrong Drive Letters

2009-11-20T02:06:00.001-08:00

Technorati tags: Windows, Ubuntu, grub, dual-boot, Sysprep, WinPE, ImageX, bug

If you…

Dual-boot Windows and Ubuntu
Have multiple Windows-visible partitions
Use WAIK / OPK tools such as Sysprep, ImageX and WinPE

…then you may find…

Sysprep fails before it completes
WinPE “sees” partitions with incorrect drive letters

The impact can be severe; finding you’ve built your .WIM from the wrong partition, having Sysprep ruin both the .WIM you harvest plus the reference system you’d built, etc. Attempts to maintain the Windows installation via WinRE or “just” re-install Windows may fail, too; I haven’t tested those scenarios.

The fix is to make sure the Windows boot partition is set as active in the partition table before you apply Sysprep or attempt access from WinPE, WinRE, OS installation disk, etc. You can do this after Windows and Ubuntu have been installed; it won’t affect these, or how grub works.

The cause is a combination of the way grub works (which bypasses the normal MBR “boot the partition that is set as “active” code logic) and the way Microsoft code assigns drive letters to Windows-visible partitions and logical volumes.

Standard MBR logic

The Master Boot Record (MBR) is the first sector of the physical hard drive, and acts as an extension of the system BIOS. It exists outside of any OS, running as it does before any particular OS has come into effect.

The standard MBR contains a partition table defining up to 4 partitions, one of which may be flagged as “active”. The standard MBR code logic is to look for the (first?) active partition and chain into code within the first sector of this space. At this point, the system phase of the boot process ends, and the OS phase begins.

How grub works

The grub boot manager adds some initial code to the MBR which links to the bulk of its code within the Ubuntu partition. At boot time, this modified MBR code will always chain into the rest of grub, irrespective of which partition entry in the partition table is set as “active”. The partition table is still referenced to find partitions, but the “active” setting is now ignored, and is thus irrelevant.

You may assume that the partition you booted via grub will be set as “active” in the partition table, but this is not the case; grub (at least grub 2, as contained in Ubuntu 9.10) does not update the “active” flag status according to what you booted last, even if set to default to this on next boot.

How Microsoft assigns drive letters

Microsoft OSs can “see” two groups of partition types; primary partitions that may be bootable and define a single volume, and an extended partition type that is not bootable but can contain multiple logical volumes. Each volume contains a single file system and is typically assigned a single drive letter.

Drive letters have validity only within a Microsoft OS. In the absence of “remembered” settings within that OS, they are assigned as follows…

A: and B: reserved for legacy diskette drives
For each physical hard drive…
Assign ascending letters to each “active” primary partition
…next drive until all done
For each physical hard drive…
Assign ascending letters to each logical volume in extended partition
…next drive until all done
For each physical hard drive…
Assign ascending letters to each “inactive” primary partition
…next drive until all done

For example, if you have an NTFS primary partition and an extended partition containing three logical volumes, these will be lettered as C:, D:, E: and F: if the primary is set as “active”, and F:, C:, D: and E: if the primary is not set as “active” – and so…

Here comes the pain

When Windows boots off the hard drive, it can override the above logic in two ways.

Firstly, it is aware of which partition it booted from, and which volume contains the bulk of its own code; these drive letters are recorded within the OS and can’t be changed.

Secondly, it remembers drive letters assigned to volumes it has “seen” before. Unlike the letters for boot and OS volumes, these can be changed by the user, causing new values to be “remembered” and applied on subsequent boots.

But when you don’t boot this OS code, e.g. you boot WinRE, WinPE or the OS installation disk instead, then all those remembered settings do not apply. I suspect Sysprep applies fresh logic during its processing as well, thus breaking its assumption base and causing it to fail.

Further, one may not be aware that the “active” flag status is an variance with boot history, and therefore assume that because you last booted Windows, that Windows partition will be the one currently set as “active”. But that is not what happens when grub is in effect.

Best practices

I would suggest the following, to reduce these sort of risks…

1. Always do an image backup prior to Sysprep

Sysprep can be as destructive as “just” re-installing Windows, or shifting/resizing existing partitions. In practice, I have far higher destructive failures with Sysprep than repair installs of XP, over-old OS version upgrades and partition management, all of which have been safer than Service Pack installs. So if you would always backup before doing those sort of things, then all the more so to backup before Sysprep.

Unlike Win9x and older Microsoft OSs, accurately copying every single file from one drive to another will not result in a bootable system, even if the drives and partitions are identical in size and you also copy over PBR contents that exist outside the file system.

That is why you have to do a partition image backup (e.g. from BING boot, using Drive Image from Bart boot, etc.) to preserve your “undo” trail.

2. Check that the Windows primary is set as “active”

This should now be added to your sanity-checks before signing off on a system build, running Sysprep, harvesting .WIM images from WinPE, etc.

If you have a WinRE installation set up to boot in the event of Windows boot failure, then it may be important for the correct partition to be set as “active” at all times.

3. Apply descriptive names to disk volumes

I apply the names “C-Drive”, “D-Drive” etc. to partitions and volumes as I create them in BING, so that these are the names I will see when working in BING to manipulate them as partitions.

BING writes these names into the boot record of the volume, whereas the name you apply in Windows is held as a Volume Label entry within the root directory of that volume. So you can have “pretty” names in Windows, Bart CDR boot, etc. and accurate names in BING.

My own practice is to choose “pretty” names that happen to start with the expected drive letter, so I get a quick visual sanity-check before operating on them in Windows. For example, if I see “Core” is C: but “Data”, “Extras” and “Factory” are E:, F: and G:, then I know something’s gone wrong and should be fixed before I generate new paths based on these wrong letters. I’d know to look for an optical drive or other intruder that has become “D:”, and fix that.

How this was tested

I tested this on new PCs build with the following hardware:

Intel “GoldTree” G43 chipset motherboard, latest BIOS applied
E6300 processor, VT enabled in BIOS (is off by duhfault)
2 x 2G = 4G DDR2-800 Kingston Value RAM
S-ATA Seagate 1.5T hard drive as S-ATA 0
S-ATA LG DVD writer as S-ATA 3 (last)

Partitions and OSs were:

30G Ubuntu 9.10 partition (not visible to Windows)
4G Ubuntu swap partition (not visible to Windows)
64G primary partition, Windows 7 64-bit, as C:
Extended partition containing FAT32 logicals D:, E: and F:
MBR contains grub 2 as installed with Ubuntu 9.10

Two PCs were tested, one with Home Basic and one with Pro as the Windows 7 edition, both being DSP (small OEM) installations. The grub menu was set to default to the OS that was booted last, and this was always Windows during these tests.

BING was used to create and manage partitions (unlike Windows, can format FAT32 larger than 32G) and was not installed as boot manager.

Test procedure:

BING boot, image backup Win7 primary to logical E:
Set Win7 primary as “active”
Boot hard drive; grub defaults to last selected (Windows), OK
Boot Windows; works, drive letters OK
Boot WinPE; what should be C: D: E: F: seen as C: D: E: F:, OK
Boot Windows, run Sysprep; works OK
BING boot; now…
Set Ubuntu primary as “active”
Boot hard drive; grub defaults to last selected (Windows), OK
Boot Windows; works, drive letters OK
Boot WinPE; what should be C: D: E: F: seen as F: C: D: E: - Fail
Boot Windows, run Sysprep; fails before post-processing boot
Windows is now not functioning, and remains so after reboot - Fail

In each case, Sysprep was run without answer file or CLI parameters; OOBE was selected, Generalize was checked, and Reboot selected as the post-processing action.

XP Blank Desktop, No Task Manager, No UI

2009-11-17T00:36:00.001-08:00

Technorati tags: XP, boot, bug

I hit this failure pattern in the context of doing an XP SP3 repair install over XP SP3 with IE 8 installed, and it may be that this is a generic issue. Searching the web did not find a solution, which is why I’m writing this.

The fix is to install IE 8 from Safe Mode.

Failure pattern

Windows XP boots to the desktop, showing wallpaper, but nothing else; no icons, Taskbar, Start button, etc. Mouse pointer present and moves OK, and the “lock” keys toggle the appropriate keyboard LEDs, so the system is still running. Safe Mode works OK.

Pressing Ctl+Alt+Del does not bring up Task Manager, pressing Alt+Tab or the Flag key does nothing, and pressing (but not holding) ATX power off does not initiate a shutdown. Pressing the case Reset button forces a hard reset and holding down ATX power button forces ATX “power off”; both cause some file system damage due to bad exit with files open for writes.

Note how this failure pattern differs from some others that are more common; no icons but UI elements present (desktop properties setting to hide icons, icons unselected, etc.), other “shell” failures where Ctl+Alt+Del and ATX power press still work, and malware effects that specifically knock out Task Manager while leaving the desktop UI functioning.

Typical Scenario

It is often necessary to do a “repair install” of Windows XP if one has changed core hardware that breaks compatibility with XP’s pre-PnP code base. This is what happened to me, when I had to replace a dead motherboard with a different one, even though this was based on the same chipset and the hard drive interface remained the same.

In addition, folks often “just” re-install Windows for all sorts of problems, even when there are cleaner ways to fix the problem, and/or when doing so is likely to fail. So I’m surprised there hasn’t been solutions visible via Internet search for this issue – if indeed it is as generic as I suspect it may be.

A few weeks earlier I’d done a similar replacement on a similar PC, using a completely different motherboard chipset. In that case, Windows booted just fine, did a bit of PnP device detection and driver installation, tossed its Activation cookies out of the cot, but was ultimately fine. But in the second case, I had the half-expected STOP BSoD error (“Windows has been shut down to prevent damage to your computer”) earlier during the boot process.

What didn’t work

Writing to an at-risk system, especially installing new software, can often make things worse – so installing IE 8 was far from the first thing I tried.

The PC had already done “the prelim”; RAM, motherboard capacitors, hard drive, file system were all OK and malware had been managed from a Bart boot, and the C: partition had been backed up as a partition image using BING.

First, I hunted down and removed all startup items and drivers for old hardware, working both from Safe Mode and Bart boot. No joy.

Because Safe Mode worked OK, I suspected a shell integration factor, so I tried setting the shell to Cmd.exe and starting Windows normally. This failed in the same way; no Cmd.exe window appeared, and ATX power, Ctl+Alt+Del etc. still didn’t work. Disabling shell extensions using Nirsoft Extension Viewer didn’t work either.

Then I thought there may be a problem with launching the shell, so I edited a batch files run via an existing Task so that it launched Explorer.exe instead of doing what it had done before. I’d noted this Task running behind the dead desktop, but from Safe Mode one cannot create new Tasks properly – some properties can’t be set, such as “run only when logged on”. That’s why I edited the batch file of an existing Task, rather than creating a new one. No joy, again.

The fix

Re-installing Windows often causes problems when bundled subsystems (e.g. Windows Media Player, Internet Explorer) are forced back to older versions. With that in mind, I tried re-installing IE 8 from Safe Mode, half expecting the usual “Windows Installer is not running” failure pattern. Much to my surprise, not only did the installation of IE 8 from Safe Mode work, the next boot brought up a functioning shell in normal Windows mode. Problem solved!

Vista UI Annoyances

2009-05-26T10:12:00.001-07:00

Technorati tags: Vista

When a user interface has different behaviours, and you can’t predict which one will arise, it can drive you nuts. Sometimes this is due to cues it takes from material you haven’t seen yet, and sometimes there’s something you need to do slightly differently to select one or other behaviour – but the difference in what you do is too subtle to learn.

File operation, multi-selection, or re-ordering?

This was always a pain in XP’s Start Menu; you try to drag an item to re-order it, and it doesn’t go with the mouse because for some reason the OS didn’t know that’s what you wanted to do. So I’d stamp the mouse button on the thing, hold still with button down for a while, then drag smartly while staying on the menu – trying to make my gestures and timing as clear as possible. No joy; sometimes it works, sometimes not. Then I tried making the first move sideways vs. up or down, making the start of the move gradual vs. sudden, and I still could not get consistent results.

In Vista, the problem sprawls over to all folder views, making the problem that much more annoying as it now pervades the whole shell. Even if you deliberately choose List view in an attempt to avoid useless icon positioning info clogging up the registry, Vista still seems to remember item positioning, as imposed via dragging within the pane.

The effect is the reverse of the Start Menu context, because usually in the shell I’m trying to select a large number of items by “lasso’ing” them, or move one or a selected wad of items from one pane to another. Sometimes I get what I want; sometimes Vista thinks I’m trying to lasso-select items when I’m trying to drag what I’ve stamped the mouse on, and other times it does the re-ordering thing, which is never what I want.

In XP, I didn’t have that confusion between lasso-selection and dragging items. As long as I started by lasso-select from an “empty” point in the folder, I’d know I’d get lasso behaviour, and not drag-and-drop behaviour.

But there’s something different in the way Vista selects things, and that’s a problem in its own that we’ll come to later. Perhaps that difference affects this UI behaviour as well?

Letter case for drive volume names

In the days of Windows 95, to avoid the overhead of LFN directory entries for valid 8.3 names, you’d have to stick to ALLCAPS. The NT family may use other more economical cues for ALLCAPS, Sentence.Case and allsmalls names, which is one reason to be less tense about all this… so today, I usually use the letter case that I want to see, rather than try to reduce system overhead of LFNs.

This works fine for files and folders, but gets wobbly when it comes to the names used by hard drive volumes. The problem is common to both XP and Vista – it appears to be impossible to force your choice of letter case; sometimes you get ALLCAPS, other times Sentence Case.

Now volume labels are tricky things, down at the file system where they are stored. Each volume actually has two separate name locations; one is embedded within the volume’s boot record, and the other is held in the root directory as a “legacy” 8.3 entry. There’s a twist to the way that 8.3 entry is interpreted; all 11 characters are seen as one name (not as 8 character name plus 3 character extension), and lower case and space characters are allowed. This behaviour goes back as far as pre-Windows MS-DOS.

When you set the volume name via the shell, only the root directory entry is affected, and this is what is displayed if it exists. If it does not exist, the name embedded within the boot record is shown; if that is blank, you will see “local disk” instead. If you want to operate on the embedded boot record name, you can do that from BING after booting it from CDR, cancelling the install prompt, and using it in partition maintenance mode.

On the face of it, preserving letter case should be even easier than for normal files and directories, because the legacy behaviour does this even without LFNs. The shell appears to restrict itself to the original 8.3 entry, as it accepts only 11 characters as input.

But whether I use F2, right-click Rename or right-click Properties etc., I cannot impose my choice of letter case, whether I “break the rules” with spaces or not. Often I have some volumes displaying as ALLCAPS and some in Sentence case, after using the same UI methods to name all of these in the same way… very strange.

Content-sensitive folder views

I’m not the only one who hates this with a passion! When you view the items in a folder, Vista gropes those items to “smell” what type of things they are, then selects the appropriate view. AutoPlay does a similar thing when it populates the pop-up list of things you can do.

If these behaviours were restricted to clearly-defined contexts, such as defined shell folders and true audio CDs, I wouldn’t mind. The problem is the cues that Vista is using to determine the context, are far too variable and flaky – one image file doesn’t mean this is a collection of photos, and one .MP3 or .WAV doesn’t make it a music collection either. Several apps will include a few image or audio files in the same directory, yet if anything these should be handled as “mixed content”.

Vista’s guessing is as absurd as vintage Windows 95’s auto-resolution of shortcuts that point to missing targets (remember “can’t find WINWORD.EXE, should I point to SMARTDRV.EXE instead, and do so forevermore if you click OK?”). Navigate into a new Start menu folder containing the Skype icon, and it will always be shown as thumbnails view; other Start menu folders containing other icons typically look “normal”. Bizarre.

There are safety aspects to this as well – when I view a folder, it may be because I know there’s malware in there and I intend to delete files without “opening” them. Having the shell code automatically groping all this material is exactly what I DON’T want – and that applies especially to the “autoplaying” of arbitrary CDRs and external storage devices.

Dead icons for living shortcuts

This also drives me nuts in Vista, and may be related to the way that Windows Installer smurfs pointers to files and icons through CLSIDs. Specifically, shortcuts created by Windows Installer’s processing of .MSI files, will not point to the actual executable, but to a spare copy of this that is held within %WinDir%\Installer – and yes, this junk can’t be relocated off C:.

Well, all of that’s just Windows Installer; irrespective of whether that’s on XP or Vista, it’s equally flaky and tedious, e.g. prone to spontaneously demanding install disks for stuff you thought you’d already installed, and weren’t even running at the time.

What’s particular to Vista, is how icons within the Start Menu – both (All) Programs and the “recently used” and pinned lists – often flip to the generic “file not found” icon, and stay that way even if you can right-click the shortcut and re-assert the icon. Given that CLSID-based post-.MSI shortcuts preclude user UI editing of target filespecs etc., maybe this isn’t related to Windows Installer after all – then again, I’ve enough other reasons to wish Installer and .MSI to go away forever.

Is it selected or not?

Vista feels “different” to XP when one is selecting items, as well as nodding the current item through these. For example, for item1 … item5, if you’re holding down the Control key and using the arrow keys to nod the current item along, after using Space to select item2 and item4, then the appearance of these items can be confusing. The selection colours are usually quite pale, and the difference between “selected”, “unselected”, “current and selected” and “current but not selected” is extremely subtle.

In contrast, XP uses different and mutually exclusive UI techniques for selected vs. unselected (different background color) and current item (outline rectangle). Vista’s fancy 3D pastels may be pretty to look at, and thus nice for the first few minutes, but they’re hard to work through, and thus a pain for ever.

In fact, it really amuses me how we’ve “progressed” as far as monitors and GUIs are concerned. First, we used curved reflective tube monitors that picked up reflections from lights and windows, and all we wanted was a matt non-reflective screen, or better yet, a screen with a flat surface that didn’t show these highlights. Now that we have flat LCDs that don’t reflect the room back at us while we’re trying to work, we add fake highlights all over everything. Then we’re told we need higher-performance (and power-hogging) hardware so we can see these added imperfections – very strange.

Automatic Update May Force-Feed You

2009-03-05T17:13:00.001-08:00

Technorati tags: Windows Update

Here’s a fun thing to try, as re-verified tonight in XP SP3: Set Automatic Updates to “Download updates for me, but let me choose when to install them”, then when the yellow shield shows new updates are ready to install, go “Advanced”, UNcheck all of them, and ignore the prompt.

Now do the same thing with the yellow shield. See how the updates are checked again? UNcheck them again, as you did before.

Now go to the Start Menu, Turn Off Computer. Notice how the dialog box is set to install updates, with the non-icon link text to shutdown without installing them?

In tonight’s case, the updates were two; one, a self-serving “Genuine Advantage” for MS Office, and the other, something to update with Windows Live Sign-In Assistant.

I’ve debated this topic in a security newsgroup, who are gung-ho to have us consumers swallow updates immediately, even if they advise against immediately rolling updates across their own corporate network “production machines” before these are tested. Well, as a consumer, I have a network of one crucial “production machine”, I don’t have pro-grade in-house testing capabilities, and yet I don’t want my system adversely impacted either.

As it stands, you can read “Download updates for me, but let me choose when to install them” to mean “let me choose whether to install them right now, or have them forced into the system on next shutdown” or (as I did), “let me choose whether or not to install them at all”. I want my downloaded updates stored somewhere in a redirectable location (hint: Not C:) so that I can initiate installation when I choose. Yes, pre-check them in the yellow shield dialog box, but if I assert my desire to NOT install them by UNchecking them, then DO NOT install them.

Google Chrome - Born Dead?

2008-09-11T01:35:00.001-07:00

Technorati tags: Web browser, safety

Web browsers are serious risk surfaces, so there's always room for a better one - but so far, most new browsers are a lot dumber than the incumbents.

So it was with Apple's Safari, when that was ported to Windows as a beta -it was found to be exploitable within two hours of release. So it is with Google's Chrome, which should be no surprise as it uses the pre-fixed exploited code from Safari!

By-design safety

Google talk a good talk, with these security features widely quoted:

Privacy mode for trackless browsing
Each tab runs in its own context, can't crash other tabs
Tabs run in a "sandbox", can't attack the rest of the system
Updates list of bad sites from Google's servers, to spot phishing scams
Web pages can open without browser UI elements (uhh... why is this "secure"?)

My first reaction when I read this was, "wow, Google scooped IE8's feature set", given that IE8 builds IE7's to phishing filter into a more comprehensive updated system, runs tabs in separate processes so they don't crash the whole browser, and Vista runs IE7 and thus IE8 in a safer "protected mode" for well over a year now. I don't know whether Google's "sandbox" is stronger and safer than IE7-on-Vista's "protected mode", or whether either of these constitute an effective "sandbox".

Then I thought: Hang on, this is a newly-released beta, whereas IE8 has been in beta for a while now and has already been more widely released as beta 2... so who's first to offer these features?

I have to wonder why Google thinks it's a good idea to spawn web content (basically, stuff foisted from the web) as generic stand-alone windows, when we already have so many problems with pop-ups forging system dialog boxes to push fake scanners etc. Why is it considered a good idea to let sites hide the address bar, when phishing attacks so often use misleading URLs that HTML allows to be covered with arbitrary text, including completely different fake URLs?

Code safety

Google talks about a large sandboxed system to interpret JavaScript, which sounds a bit like the idea behind Java. Well, we've seen how well that works, given the long list of security updates that Sun have to constantly release to keep up with code exploits - so we'd have to hope Google are really good at crafting safe, non-exploitable code.

So it doesn't bode well, that the public beta they release is based on a known-exploitable code base, which is already being attacked, at a time when patched versions of this code are already being retro-fitted to existing Safari installations.

Why would Google not build their beta on the fixed code base? It's Open Source, and already available, why not use it? Would it have killed them to delay the hitherto-secret web browser beta until they'd adopted the fixed code? Or is the need to leverage pre-arranged hype etc. more important than shipping known-exploited code to users? And why does the fixed release still report the exploitable code base version?

Trust me, I'm a software vendor

How do you feel about vendors who silently push new code into your system and are slow to tell you what it does? Here's what Google is quoted as saying about that:

"Users do not get a notification when they are updated. When there are security fixes, it's crucial that we update our users as quickly as possible in order to keep them safe. Thus, it's important for us to not require user intervention. There are some security fixes that we'll keep quiet because we don't want to disclose security vulnerabilities to attackers"

To me, that reads like a dangerous combination of Mickey-Mouse attempts at security via obscurity, plus supreme vendor arrogance.

But wait, there's more...

Further things have come to light when searching for links for this post, such as installing in a "data" location (thus side-stepping Vista's protection for "Program Files") and a rather too-effective search that finds supposedly private things.

"Well, it's a beta", I can hear you say. That's why it's safely tucked away deeply within Google's developer site, so that only the adventurous and knowledgeable will find it, right? I mean, it's not as if it's being shoved at everyone via popular or vendor-set web pages so that it's gaining significant market share, is it?

Compatibility vs. Safety

2008-09-10T06:29:00.001-07:00

Technorati tags: Software Updates, Safety

Once upon a time, new software was of interest because it had new features or other improvements over previous versions. This attracted us to new versions, but we still wanted our old stuff to work - so the new versions would often retain old code to stay compatible with what we already had.

Today, we're not so much following the carrot of quality, but fleeing the stick of quality failure. We are often told we must get a new version because the old version was so badly made, it could be exploited to do all sorts of unwanted things. In this case, we want to break compatibility so that the old exploit techniques will no longer work!

Yet often the same vendors who drive us to "patch" or "upgrade" their products to avoid exploitation risks, still seem to think we are attracted by features, not driven by fear.

Sun's Java

I've highlighted the long-standing problems with Sun's Java before, and they are still squirming around their promise to mend their ways. In short, they may still leave old exploitable versions of the Java JRE on your system, but it's no longer quite as easy for malware to select these as their preferred interpreter. Still, you're probably safer if you uninstall these old JREs (as Sun's Java updater typically does not do) than trust Sun to deny code access to them.

Microsoft's Side By Side

Here's an interesting article on the Windows SxS (Side By Side) facility, which aims to appease software that was written for older versions of system .DLLs and thus ease the pain of "DLL Hell". This works by retaining old versions of these .DLLs so that older software can specify access to them, via their manifest.

How is that different from Sun's accursed practice?

Well, is generally isn't, as far as I can tell, until a particular exploit situation is recognized where this behaviour poses a risk. The current crisis du jour involves exploits against GDIPlus.dll - yep the same one that was fixed before - and the patch this time includes a facility to block access to old versions of the .DLL, leveraging a feature already designed into the SxS subsystem.

The Most Dangerous File Type Is...

2008-09-05T20:49:00.001-07:00

Technorati tags: Malware, Safety

The most dangerous file type is... what?

Well, you pass if you said ".exe", and get bonus marks for ".pif, because it's just as dangerous thanks to poor type discipline, and more so because of poor UI safely that hides what it is". But today's answer may be neither.

By the time a code file lands up on your system, there's a chance your antivirus will have been updated to know what it is, and may save the attacker's shot at goal. But a link can point to fresh malware code that's updated on the server side in real time; that's far more likely to be "too new" for av to detect, and once it's running, it can kill or subvert your defences.

We need to apply this realization to the way we evaluate and manage risk, to up-rate the risk posed by whatever can deliver Internet links. Think "safe" messages without scripts or attachments, and blog comment spam (including the link from the comment poster's name).

Think also about how HTML allows arbitrary text to overlie a link, including text that looks like the link itself. This link could obviously go to www.bad.com, but it's less obvious that www.microsoft.com could go there instead. Then think how HTML is ubiquitously tossed around as a generic "rich text" interchange medium, from email message "text" to .CHM Help files.

Bart Plugin for Spybot 1.6

2008-08-13T10:31:00.001-07:00

Technorati tags: Bart PE, Bart plugin, Spybot, maintenance OS

See previous post about the new version 1.6 of Spybot SD and its issues. I've updated my Bart plugin (tested with XP SP2 code base, Bart Builder 3.1.3) to address these, and offer it here, along with .REG for control in Windows.

To use the plugin, do this:

Navigate into your Bart Builder plugin folder
Create new folder called SpybotSD and enter it
Copy this post's plugin files to this location
Create a subfolder Files within this location and enter it
Copy the installed Spybot 1.6 subtree contents into here

The plugin is written with these assumptions and dependencies:

Standard Bart PE Builder with nu2menu as shell
Cmdow utility in Bart included Bin folder (not essential)
Paraglider's RunScanner plugin in plugin\RunScanner

Cmdow

Cmdow hides windows for processors, and I use it to hide the .CMD launcher; it's purely cosmetic, so if missing, the plugin will still work. Because Cmdow can be dropped on systems and used maliciously, many scanners will detect it as a "potentially unwanted program", and fair enough!

RunScanner

RunScanner allows registry-aware tools to run relative to an inactive set of hives, rather than those of the booted OS. Spybot has native awareness of this situation, so theoretically doesn't need RunScanner, but I find I get better detections if I use it anyway. If RunScanner isn't present, you'd have to revise the .INF and .XML for it else it won't work.

SpybotSD.inf

This determines how Spybot 1.6 is integrated into the Bart CDR at build time.

; spybotsd.inf
; PE Builder v3 plug-in INF file for Spybot - Search & Destroy by Safer Networking Ltd.
; Created by Patrick M. Kolla, Jochen Tösmann and modified by cquirke for Spybot 1.6

[Version]
Signature= "$Windows NT$"

[PEBuilder]
Name="Spybot - Search & Destroy"
Enable=1
Help="spybotsd.htm"

[WinntDirectories]
a="Programs\SpybotSD",2
b="Programs\SpybotSD\Dummies",2
c="Programs\SpybotSD\Excludes",2
d="Programs\SpybotSD\Help",2
e="Programs\SpybotSD\Includes",2
f="Programs\SpybotSD\Languages",2
g="Programs\SpybotSD\Plugins",2

h="Programs\SpybotSD\HelpHTML",2
i="Programs\SpybotSD\HelpHTML\css",2
j="Programs\SpybotSD\HelpHTML\html",2
k="Programs\SpybotSD\HelpHTML\images",2

[SourceDisksFiles]
*.cmd=a,,1

files\blindman.exe=a,,1
files\SDMain.exe=a,,1
files\SDUpdate.exe=a,,1
files\SDWinSec.exe=a,,1
files\SpybotSD.exe=a,,1
files\TeaTimer.exe=a,,4
files\Update.exe=a,,4
files\advcheck.dll=a,,1
files\aports.dll=a,,1
files\DelZip179.dll=a,,1
files\SDHelper.dll=a,,4
files\Tools.dll=a,,4
files\messages.zres=a,,1
files\Tools.dll=a,,1
files\sqlite3.dll=a,,4

files\Dummies\*.*=b,,1
files\Excludes\*.*=c,,4
files\Help\*.*=d,,4
files\Includes\*.*=e,,1
files\Languages\*.*=f,,4
files\Plugins\*.*=g,,1

files\HelpHTML\*.*=g,,4
files\HelpHTML\css\*.*=h,,4
files\HelpHTML\html\*.*=i,,4
files\HelpHTML\images\*.*=j,,4

[Software.AddReg]
0x4, "Safer Networking Limited\Tweaks", "DisableTempFolderCleaning", 0x1
0x1, "Paraglider\RunScanner\SpybotSD.exe", "HKLM", "Software\Safer Networking Limited\Tweaks"

[Append]
nu2menu.xml, spybotsd_nu2menu.xml

Ensure that when you copy and paste these files, that they are free of HTML tags and formatting junk, and that long lines (e.g. the two lines in the last section) are not broken. The above differs from Safer Networking's plugin for 1.5, in that:

It includes new code file sqlite3.dll
It suppresses automatic temp file clearance
It persists the above setting through RunScanner

The last is useful, so you don't have to use non-zero /t parameters in an attempt to delay registry redirection until Spybot has checked for the "disable temp clearance" setting.

SpybotSD_nu2menu.xml

This integrates Spybot 1.6 into the Bart menu system, and is referenced from the .INF during build time.

<NU2MENU>
<MENU ID="Programs">
<MITEM TYPE="ITEM" DISABLED="@Not(@FileExists(@GetProgramDir()\..\SpybotSD\SpybotSD.exe))" CMD="RUN" FUNC="@GetProgramDir()\..\SpybotSD\SpybotSD.exe">Spybot 1.5.2</MITEM>
</MENU>
</NU2MENU>

You may change this to strip references to RunScanner, relocate it to a different menu flyout etc. or if you're fed up with disordered menus, you may simply leave out this file (; comment it out in the .INF) and add your reference directly to plugin\nu2menu\nu2menu.xml - once again, watch out for long lines; there is in fact only one line between the MENU ID and /MENU tags.

SpybotSD.cmd

This launches Spybot 1.6 from the nu2menu entry at runtime.

@Echo Off

SetLocal

Set Debug=
Set Prog=SpybotSD.exe
Set Launch=%~dp0..\RunScanner\RunScanner.exe
Set Opt=/t 0

If Not Defined Debug (
Cmdow @ /HID
%~dp0..\..\Bin\Cmdow @ /HID
) Else (
Title Debug
Echo.
Echo ProgDir %~dp0
Echo Prog     %Prog%
Echo Launch   %Launch%
Echo Opt      %Opt%
Echo.
Pause
Title %~dp0%Prog%
)

If Exist "%~dp0Files\%Prog%" Set ProgDir=%~dp0Files\
If Exist "%~dp0%Prog%"       Set ProgDir=%~dp0
If Defined ProgDir (
If "%SystemDrive%"=="%~d0" (
    Start %Launch% %Opt% %ProgDir%%Prog%
) Else (
    Start %ProgDir%%Prog%
)
) Else (
Title Error - target executable not found!
Echo "%Prog%" not found in %~dp0 or %~dp0Files\ - abort!
Pause
EndLocal
Exit /b 1
)

If Defined Debug (
Echo.
Echo Done!
Echo.
Pause
)

EndLocal

Exit /b 0

You can edit this to strip out the "debug" part (define the Debug variable to enable it), as well as references to Cmdow and RunScanner. By changing the variables, you can use this for other "easy" tool plugins (e.g. HiJackThis).

The logic goes as follows; if boot drive is same as where we are, then we're Bart-booted and need to apply RunScanner redirection, else we're not, and can run the tool directly. This logic will also not use RunScanner if run from a WinPE 2.0 boot disk, which is OK with me as I don't know how safe RunScanner is for Vista hives.

An extra bit of logic is applied to deriving the path to the tool, so that the .CMD will work when run from the pre-build subtree. This is also why the .XML uses relative "GetProgramDir()\..\" paths, rather than the more commonly used "GetProgramDrive()\Programs\" paths that break in the pre-build or pre-iso environments.

Windows .REG

You can also control some of Spybot's potentially unwanted behaviours via .REG in Windows, similar to the Software.AddReg section in the .INF above:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Safer Networking Limited\Tweaks]
"DisableTempFolderCleaning"=dword:00000001

[HKEY_LOCAL_MACHINE\SOFTWARE\Paraglider\RunScanner\SpybotSD.exe]
"HKLM"="Software\\Safer Networking Limited\\Tweaks"

The second part of the above will pre-load appropriate settings for a Bart session using RunScanner, in case the RunScanner's parameters cause it to read its settings from the hard drive's hives.

Some settings can be changed interactively, e.g. disabling the intrusive Tea Timer feature, while others have to be excluded at the time of installation. One of the latter, is the right-click context menu action to scan using Spybot, which annoyed these folks who offer this fix:

Windows Registry Editor Version 5.00

[-HKEY_CLASSES_ROOT\*\Shell\sdfiles]

[-HKEY_CLASSES_ROOT\Folder\shell\sdfiles]

[-HKEY_LOCAL_MACHINE\SOFTWARE\Classes\*\Shell\sdfiles]

The * association is applied to all things, hence all things can be right-clicked and scanned. There's an Undo .REG in the same post in that thread.

Spybot 1.6 and Bart PE

2008-08-12T07:58:00.001-07:00

Technorati tags: Bart PE, maintenance OS, malware

Malware scanners tend to focus on resident protection rather than intervention and clean-up, but Spybot has always had a clue there. Not only does Spybot explicitly support Bart PE as a formal scanning platform, it can also be aware of inactive registry hives, e.g. if you were to drop an ?infected hard drive into a Windows host system to clean it from there.

Bart has a plugin facility to integrate tools, and whenever there's a new version of a plugged-in tool, there may be changes required, or new unwanted behaviours to work around. Such is the case with the new Spybot 1.6

Spybot 1.6 plugin changes

A Bart plugin is a set of files that control how a program is integrated into a Bart CDR. Build-time instructions are defined in an .inf, menu integration via an nu2menu.xml, runtime control via a .cmd (if needed), and human documentation via an HTML file.

The .inf defines what files are to be copied to the CDR and where they are to be located, in the SourceDisksFiles and SourceDisksFolders sections. If you've used SourceDisksFiles to explicitly name every file from within Spybot 1.4 or 1.5 to be copied to CDR, and you then drop in the Spybot 1.6 file set and build a new Bart disk, then you'll find Spybot will fail to launch from the disk.

If so, you can fix this by adding a line to include sqlite3.dll, which is a new file not present in earlier versions of Spybot SD. Or you can use wildcard syntax to include all dll files, i.e. *.dll as files to be included.

Unwanted behaviour

Spybot 1.6 has a controversial new feature; it deletes Temp files when it starts up. This is "controlled" by a 6-second dialog box that appears as Spybot starts up (so if you start it and walk away, you'll miss it) and defaults to "Yes, delete temp files".

This is a bigger problem within the Bart environment, which often has troublesome graphics due to unrecognised display chipsets. In my first Bart session with Spybot 1.6, I expected the dialog, but it appeared with blank buttons. By the time I checked out what button was what, testing on another PC, the 6 seconds were up, and I'd lost material I'd have preferred to include in further malware scans.

There is a rather obscure fix for this, which I will add to my Bart plugin's .inf file, using one of the registry modification sections. If using the RunScanner plugin to launch Spybot (should not be required, as Spybot "knows" about such needs), then you'd want to delay the RunScanner redirection until this value had been read by Spybot after starting up - else it will look for it in the inactive (target) hives instead.

Lazarus of Bad Hard Drives

2008-08-09T01:57:00.001-07:00

Technorati tags: PC maintenance, Data Recocery

Here's a failure pattern worth keeping in mind:

Unbootable OS
Attempts to access hard drive lock up or fail
Hardware diagnostics show or imply bad sectors
You image the raw partition to a good hard drive
Data access and even OS bootability miraculously OK

This is Lazarus of Bad Hard drives, as opposed to Lazarus of Bethany.

What's happening here, is that effects from deeper abstraction layers are creating what appears to be unfixable problems in higher layers. What is counter-intuitive is that fixing the underlying layer can fix the upper layers too, i.e. that the state of these layers may not be irreparably botched by the lower-layer failure.

So don't give up hope, if you hit the first three items in the list above.

This Hard Drive, Which PC?

2008-07-27T14:53:00.001-07:00

Technorati tags: PC maintenance

Let's take two unrelated ideas and draw them together...

If you take Fred's brain and transplant it into Martin's body, have you performed a brain transplant on Martin, or a body transplant of Fred?

If the processor is your computer's brain, then the computer's hard drive is your mind.

Do you think the second sentence should have been "...the computer's hard drive is the computer's mind"? Both statements are true, but from your perspective, the first may be more important.

Why might this matter?

The practical importance of this arises if you take a hard drive out of a PC, and then come back after a while with that hard drive in your hand, and a collection of PCs without hard drives on the bench.

It's easy to tell who the hard drive belongs to, because the files it contains will be full of cues. But which PC belonged to that particular user? Less easy, because all information uniquely linking that computer to the user is on the hard drive. Unless you have kept some other record, e.g. serial number tracking, ownership sticky notes, arrival photos etc. you could have a problem.

Solving this problem

On old PCs that don't auto-detect hard drives, you can examine CMOS to look for CHS geometry etc. that matches the hard drive in your hand. But new PCs generally don't persist CHS or other hard drive parameters in CMOS; they re-detect such devices on POST instead.

You can examine the hard drive's files to look for links to the particular hardware, e.g. an OS product key that matches a case sticker, or a collection of device drivers that map to the rest of the PC hardware.

That is not as easy as looking for cues to the user of the PC; you may have to bind registry hives and look for cues in there. You could do that implicitly by CDR-booting into Bart and using RunScanner to wrap Regedit or other tools so they map to the hard drive installation's hives, or you can do that explicitly by running Regedit from any suitable host OS, and manually binding the hard drive's hives under HKLM.

Don't boot in the wrong PC!

Whichever approach you use to examine the hard drive, it's crucial not to allow the hard drive to boot in the wrong PC, in all but the most trivial of OSs. DOS would be safe, but anything more complex is likely to go wrong for various reasons.

Win9x will use Vmm32.vxd as the core driver set upon which all other drivers are loaded, and that core driver code was derived from the particular PC hardware that was in effect when Windows was installed on it. If it's incompatible, then the PC will crash before the OS has reached sufficient "consciousness" to Plug-n-Play.

This is a common crisis when changing the motherboard under a Win9x system, and can be solved by rebuilding a new Vmm32.vxd appropriate to the new hardware. Yes, there's one on the installation disk, but it's an empty stub upon which Windows Setup builds the "real" file at install time.

You could rebuild Vmm32.vxd by re-installing the Win9x over itself, but that is messy; breaks patches and subsystem upgrades, loses settings, and so on. Or you could do a fresh install of Win9x on a different hard drive, harvest the new Vmm32.vxd from there, and drop that into the "real" hard drive from DOS mode.

Having got this far, Windows should now boot to the point that Plug-n-Play can detect the rest of the new hardware and nag you for drivers. Expect problems from hardware-specific code that was not installed as drivers, and is thus not disabled when PnP detects the hardware as no longer present - think packet-writing CD software, modem fax and especially voice software, hardware-specific Properties tabs added to Display Settings, etc.

XP, Windows 2000 and older NT-based OSs may fail in a similar way, though the installation-time hardware-specific code file will he the HAL (Hardware Abstraction Layer) rather than Vmm32.vxd - same difference, in other words.

You will probably have to do a repair installation to fix this, with similar impact as installing a Win9x over itself. I'm not sure if cleaner fixes would work.

XP may have another surprise for you, if you aren't BSoD'ed by a HAL compatibility STOP error; Windows Product Activation may see the changed PC as "too different", and trigger the DoS (Denial of Service) payload.

Vista doesn't have the same HAL mechanism as earlier NT, so it may start up without a STOP error, but you'd still have the Product Activation payload to contend with - as you would with MS Office versions XP and later, regardless of OS.

I can't tell you how Linux, BSD or MacOS (pre- or post-BSD) would fare when booted in the "wrong" computer, but I suspect similar issues may apply. Aside from artificial crises caused by deliberately malicious product activation code, you may still have the core problem of needing hardware-specific information to boot the logic that can respond to altered hardware.

Safety First

If you are not certain that you're putting the hard drive in the correct PC, then you should back up C: in such a way it can be restored as a working OS, if mistakes lead to a PnP or product (de)activation mess.

For a Win9x, that's as simple as copying off all files from the root directory, all of the OS subtree, and (easiest) everything in "Program Files". For best-practice on WinME, you should preserve the _Restore subtree too. These are the only contents of C: that should be affected by the OS "waking up" in the wrong PC.

For NT, it's messier, because Windows 2000 and later (I haven't tested earlier NT) will not boot after a file-copy transfer from one hard drive to another, even if you meticulously include all files. So you're obliged to do a partition-level copy (e.g. save C: as a partition image via BING) to maintain undoability.

Should You Detect Old Malware?

2008-07-25T02:24:00.001-07:00

Technorati tags: Malware, Safety, Security

We've gone from thinking of software as a "durable good" to evolving under selection pressure. This certainly applies to malware and blacklist-driven scanner countermeasures, which are assumed to become either extinct or irrelevant over time.

Who needs old scanners?

You may want to keep an old scanner if it still detects stuff other scanners can miss, or if it was the last version that ran in your environment - though as an "extra" on-demand scanner, not as sole resident protection, of course.

For example, Kaspersky's CLI scanner no longer runs under Bart, and I haven't adapted AdAware 2007 (which I don't particularly like) to run in Bart either. There are still manual updates for AdAware SE, so that's still "current", and Kaspersky CLI still works in Safe Cmd. However, I may want the safety of formal scanning with Kaspersky CLI, and for that, I'd need the last "old" version and updates that still worked from Bart.

Another example is McAfee's Stinger. Just as an "old" Kaspersky CLI may find stuff other updated scanners will miss, so it is with the even older Stinger - it's particularly good at catching TFTP-dropped malware and some bots, both of which are likely to be found in an NT that has not patched RPC against Lovesan et al.

F-Prot's DOS and Win32 CLI scanners are also discontinued, i.e. no further updates, but are still useful. Specifically, these scanners will often detect "possibly new version of ... Maximus", and sometimes that rather loose and false-positive-prone detection still finds things others miss. These scanners also find other false positives unrelated to Maximus, so handle with care.

Does old malware matter?

Firstly, you may encounter vintage malware on vintage systems and diskettes (e.g. boot sector infectors on old DOS or Win95-era PCs, old MS Office macro infectors in "documents" from old systems). Malware of that era were mostly self-contained and fully automated, and often had destructive payloads, so they will still bite... so you'd want to detect them.

Secondly, think of the spammer equivalent of the guy who still uses a PC built from old parts running MS Office 2000 on Windows 98, because these old feeware programs pre-date automated defence against piracy - i.e. no software budget.

A less-obvious feature of botnets is that those who own them, don't want folks controlling them for free. So if you want to send spam through a modern botnet, you will probably have to find someone and pay them.

On the other hand, old bots that are still in the wild, may have been cracked so they can be operated for free - or may simply pre-date the rise of malicious info-business and thus lack modern mechanisms to block control. In which case, our impoverished spammer may use these instead - so it may still be prudent to detect and kill them off, especially in the context of poorly-patched or defended systems (e.g. unpatched Windows 2000, no firewall, outdated or missing av).

XP Repair Install

2008-07-07T12:00:00.001-07:00

Technorati tags: XP, install

Re-installing Windows XP isn't a good idea as a blind first step in troubleshooting problems, but there are specific contexts where it is necessary, as the cleanest way to "make things work". One of these contexts is after a motherboard change that invalidates XP's core assumptions, typically causing a STOP BSoD on any sort of attempted XP boot (from Safe Cmd to normal GUI).

This is the situation that edgecrusher is in, as posted in comments to the previous post in this blog, and this post is my response.

Before you start

Firstly, I'm going to assume you have all the necessary installation and drivers disks, have your XP product key or retrieved this via Nirsoft Produkey or similar, excluded malware, and verified RAM overnight e.g. via MemTest86 or MemTest86+ and hard drive e.g. via HD Tune.

Make sure the edition (OEM vs. retail, Home vs. Pro, etc.) of the XP installation disk you will use for the repair install is one that matches your product key, that the disk actually has the ability to do a non-destructive install (as many OEM disks do not), and that the disk can be read without errors (as tested by copying all files to a subdir on the hard drive before you start).

It's a good idea to make a partition image backup of your XP installation before you start, using something like BING. Simply copying off every file is not enough, because unlike Windows 9x, XP will not work when copied in this way.

Also before you start, you may want to uninstall any OS-bundled subsystems that you've upgraded past the baseline of your XP installation disk, such as IE7 or recent versions of Windows Media Player. Things are cleaner and more likely to be "supported" if you uninstall these before the repair, and re-install them afterwards, plus you'll have valid entries in Add/Remove Programs should you need to uninstall them again later (e.g. as a troubleshooting step).

Several sites describe the XP repair install process, starting from how to start the process, and going on to a step-by-step slide show or providing more detail. In this post, I will mention a few specific gotchas to avoid...

137G capacity limit

If your hard drive is over 137G in size, then the Service Pack level of the Windows XP installation disk must be at least SP1 to install, and SP2 to live with. In other words, you cannot safely install XP "Gold" (SP0) on a hard drive over 137G, and should apply SP2 or SP3 over an XP SP1 installation.

If your install disk pre-dates SP1, you need to slipstream a later Service Pack into this and make a new installation disk that includes SP1 or later, built in. Your other option is to install XP "Gold" onto a hard drive smaller than 137G, apply SP1 or later, and then use a partition transfer utility to copy the partition to the larger hard drive where the partition can then be resized to taste.

XP "Gold" has no awareness of hard drives over 137G and is very likely to mess them up. XP SP1 is supposed to be safe on such hard drives, but there are some contexts where the code that writes to disk is unsafe and may cause corruption and data loss; from memory, these contexts typically apply to C:, e.g. writing crash dumps to the page file. XP SP2 and SP3 are truly safe over 137G.

F6 driver diskette

Yep, you read right; that's "diskette" as in "ancient crusty old stiffy drive"!

Most current motherboards have S-ATA hard drive interfaces that are not "seen" by the native XP code set (affecting Bart and Recovery Console boot disks as well).

The trouble is, the latest PCs often have no diskette drive, and the latest motherboards often have no legacy diskette controller. You may come right with an external diskette drive plugged in via USB. You'll also have to find and download the relevant driver diskette image and make a diskette from this, if yours is missing or unreliable.

If you use a USB keyboard, and this is not initiated at the BIOS level, then your F6 keystroke to read the driver diskette will be missed. If so, you can plug in a PS/2 keyboard... as long as your new motherboard has PS/2 sockets; the newest ones don't.

Sometimes your mileage may vary, depending on the mode that your S-ATA is set to operate in CMOS Setup. RAID and AHCI will generally not be "seen" natively by XP's code, whereas IDE mode may be. But some nice S-ATA features may not work in IDE mode, e.g. hot-swapping external S-ATA or NLQ, and changing this after XP is installed may precipitate the same crisis as the motherboard swap... requiring a repair install to fix, again.

All of this is a reason why I consider the XP era to be over, when it comes to new PCs. I appreciate how old OSs run beautifully fast on new hardware, and how attractive that is for gamers in particular - but XP's getting painful to install and maintain, and this is going to get worse.

Duplicate user accounts

Later in the GUI part of the installation process, you will be prompted to create new user accounts. You can try to skip this step (best, if that works... I can't remember if it does), or create a new account with a different name that you'd generally delete later.

But many users are likely to create a new account with the same one as their existing account, and that's likely to hurt...

The two accounts will show the same name at the Welcome screen, but both will be selectable via this UI; I have no idea what will happen if you were to force the more secure legacy logon UI, which requires the account name to be typed in.

Each account will have a unique Security Identifier (SID), which is the real "name" used behind the scenes - but you can't login with that. There will also be separate account subtrees in "Documents and Settings"; the one with the plainest name is likely to be the original, and the one with numbers or the PC name added to it is likely to be for the newly-spawned account.

At this point I'll mention another user account hassle that I generally don't see, because I avoid NTFS where I can. If you find you can "see" your old user account's data, but aren't permitted to access the files, then you may have to "take ownership" of these files from a user account that has full administrative rights.

This issue is well documented elsewhere; search and ye will find!

Broken update services

It's a given that the "repair" is going to blow away all patches subsequent to the baseline SP level of the XP installation disk you are using, unless you've slipstreamed these into your installation disk.

What's less obvious is that after you do the "repair" install, you won't be able to install updates. It doesn't matter whether you try via Automatic Update, Windows Update or Microsoft Update, the results will be the same; the stuff downloads OK (costing you bandwidth) but will not install, whether you are prompted to restart or not.

The cause is a mismatch between the "old" update code within the installation CD, and the newer update code that was controversially pushed via update itself. I can see Microsoft's logic here; if you ever wanted updates to work (e.g. you'd chosen "download but don't install", or disabled updates while planning to enable them later), then the update mechanism has to be updated - but doing so, invalidates the original installation disk's update code.

This topic is well-covered, as is the fix; manually re-registering a number of .DLLs that are needed for the update process to work.

Broken settings

It's often asserted that a repair install "won't lose your settings", and is yet waved around as a generic fix for undiagnosed problems. Part of why it sometimes works as a "generic fix" is precisely because it can and does flatten some settings, which may have been deranged to the point that the OS couldn't boot!

So if you do apply any non-default settings, you should check these to see if they've survived. I always check the following, and can't remember with certainty which ones survive and which don't:

System Restore (may be re-enabled on all volumes)
System Restore per-volume capacity limits
Automatically restart on system errors
RPC Restart the computer on failures (may survive)
Show all files, extensions, full paths, etc. (may survive)
NoDriveTypeAutoRun and NoDriveAutoRun
Standard services you may have disabled
Hidden admin shares, if you'd disabled them
Recovery Console enabling settings
AutoChk parameters in BootExecute setting
Shell folder paths
Windows Scripting Host, if you'd disabled it
Settings detail in IE, including grotesquely huge web cache
Windows Firewall settings; may be disabled if < SP2 !!
Anything else you've dared to change from duhfaults

It's particularly crucial to enable the Windows Firewall (or install a 3rd-party alternative) before letting your PC anywhere near any sort of networking, especially the Internet, if your installation is "Gold" or SP1. Not only do these dozeballs duhfault to "no firewall", they're also unpatched against RPC (Lovesan et al) and LSASS (Sasser et al) attacks, so you'd be "open and revolving".

By now, the original PoC Lovesan and Sasser worms may be extinct, but these exploits are often crafted into subsequent workaday bots and worms. You may still get hit within an hour of plugging in the network cable if so, and probably before you can pull down updates for the OS, antivirus scanners, etc.

Cause and Distance

2008-07-07T03:41:00.001-07:00

"Send this habitat module to Sirius Prime, now!"

' OK ...'

Right-click habitat module, Properties, Location tab, highlight "Earth", enter new text "Sirius Prime", press Enter. Drone work, really, but Fred just counts himself lucky at being able to find a summer job.

Sharp distance runaround

I find it interesting that despite the strange and counter-intuitive models we've developed for sub-atomic matter, we still cling to the Newtonian idea that cause is carried by force, and force involves objects banging into each other.

So we've had "The Aether" before we could get our heads around empty space, the idea that waves must travel in a medium (neatly solved by counter-generating electro and magnetic fields), and the "problem" of action at a distance that is modelled on throwing particles around.

We understand space and time as interrelated through the speed of light, so that "distance" can be envisaged in either terms.

Trapped in the mesh

Trying to resist the idea of desire-guided evolution, consider the need for senses (or sensors, if you like). We sense what we need to attain, avoid or overcome, not what we can happily ignore as irrelevant. Does hard drive S.M.A.R.T. monitor the tides of the ocean? Nope. Does the body monitor radiation levels? Nope, as this had not been relevant during the timescale when shaped by selection pressure.

We are of the universe and are unable to transcend "distance" (be it conceptualised as time or space) at will, though we have a limited ability to physically move towards or away from things. So we perceive distance as a dominant property of our environment, shaping concepts such as "cause and effect" and "the arrow of time".

But this perspective may be a platform-specific perception issue, rather than a universal truth. Perhaps if we visualize things differently - e.g. consider the distribution of mass as a constant, and "distance" as a particular parameter, then some things may snap in to focus, such as gravity as a "curvature of space".

Often a graph that has a shape that is hard to grapple with, becomes a tame line drawing when the scaling of an axis is changed; certain problems, such as shapes that tend towards but never reach zero, may resolve themselves. So it may be with "distance".

After writing this, I found the articles I've linked to, along with this one. It would probably be more enlightening to fan out from here than to read the post you have just finished reading :-)

XP SP3 "Stuck" Activation Dialog

2008-06-28T03:54:00.001-07:00

Technorati tags: XP SP3 bug activation

You may see this failure pattern:

Windows XP (SP3) demands activation
You get the first dialog of the activation wizard
But no matter which option you choose, Next doesn't

Specifically; this is the first dialog page of the activation wizard, from which you choose "activate via Internet", "activate via telephone" or "no, I'll do it later". When you press the Next button, the button appears to depress fine, but when you let it go, the dialog stays where it is.

If you were trying to activate by phone, then that means you don't see the list of locations to call, or the key to read to the call center if you do call. So when you call the activation center, the first thing they ask you to do ("please read me your installation ID"), you can't do, and frustration follows.

Context details

I've seen this once, in the following context. A PC had suffered hard drive failure, and was also in need of RAM upgrade and software updates. So I first repaired the hardware by imaging to a good hard drive, added more RAM, replaced the duff CD-ROM drive with a working one.

Then I did "the prelim"; MemTest86 to verify RAM, Bart CDR boot to verify HD via HD Tune, file system checks OK, formal malware scans OK.

Next, I boot into Windows, and am not too surprised when it tells me I need to activate, as the hardware has "changed too much". I deferred this, and did what I usually do when updating XP systems this month:

get off all networks and Internet
uninstall free AVG 7.5
uninstall Internet Explorer 7
move all $.. folders from OS subtree to another HD volume
defrag to consolidate free space
apply XP SP3 from offline installer
verify firewall is on
install free AVG 8
connect to Internet so AVG 8 can update
upgrade other software; Java, Acrobat Reader, Firefox etc.
allow Automatic Update to pick up IE7 and other updates
attempt activation before applying OS and IE7 updates

During this process, I restarted Windows several times for various reasons, but the activation dialog would not work. It only worked after I applied the pending Automatic Updates; then after the restart that followed, activation was fine.

Suspected cause

I suspect that pending updates cause the activation dialog to "stick". This may apply specifically to XP SP3 or be a general XP issue that I had not encountered until now, as I seldom (if ever) have activation demands and pending updates at the same time.

That situation can arise in the context of installing XP SP3, because:

you want to uninstall IE7 (or IE8 beta) before applying SP3
Windows Media Player falls back to old version
you can't install Media Player 11 as it won't "validate"
you can't install IE7 from pre-downloaded file
you can't use the Update web site until you activate

So you rely on Automatic Update to feed in the patches you want, but may feel the need to defer installation of these until you've activated. This applied in my case, because a lot of the updates were for IE6 which I intended to replace with IE7 anyway - so before applying updates, I wanted to install Media Player 11 and IE7, so that the updates I downloaded and applied would be "after" these.

If my hunch about the cause of this failure pattern is correct, then this combination of circumstances can create a "deadly embrace" of cross-dependencies; can't activate until updates are applied, but user doesn't want to apply updates until the system is activated.

Can't install IE7 on XP SP3?

On "you can't install IE7 from pre-downloaded file"; this seems to be a different XP SP3 issue.

Usually, I can at least initiate the IE7 install from a pre-downloaded installation executable, though this needs to be online so it can pull down updates to IE7 as part of the installation process. But this fails after XP SP3 has been applied; instead, one has to induce an IE7 install via Automatic, Windows or Microsoft Update. Sometimes it's offered as a critical update, other times not?

When Add/Remove Doesn't Remove

2008-04-15T16:32:00.001-07:00

Technorati tags: Uninstaller, XP, bug

What do you do when you go to XP's Control Panel, Add/Remove Programs, find the software you want to remove, and the entry has no Remove button on it?

I found an answer in a forum thread, as follows...

HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{Program Name}

If NoRemove is set to 1, the Remove button will be unavailable
If NoModify is set to 1, the Change button will be unavailable

Note that as the {program name} may be a CLSID, you may need to search for the product name (e.g. "Intel Audio Studio", using what you saw in Add/Remove) to locate the correct entry in which to find the relevant NoRemove setting.

Intel Audio Studio

In my case, I'd had to replace a failed Intel 945G chipset motherboard with a new Intel G33 chipset motherboard with attendant processor upgrade. XP died on a BSoD STOP error on all boots, as expected (a self-serving product fragility that helps limit "license creep") so the next step was to "just" do a repair install... but that's another day's blogging.

The other motherboard came with Intel Audio Studio, which installs and uninstalls along with the sound drivers. But when the motherboard changes, the old sound device vanishes from Device Manager, so you no longer have an UI from which the device (and thus drivers, and thus associated bundleware) can be removed.

Hence the problem: Add/Remove Programs has an entry for Intel Audio Studio, but that entry has no Remove button. It "doesn't need it" (so implies the text within the entry) because it is "installed and uninstalled with the drivers".

The meta-bug behind the bug

Alun Jones said "Don't solve problems, solve classes of problems", and by meta-bugs, I mean the classes of problems behind the bugs you step on one by one.

Have you noticed how few USB-interfaced hardware vendors create driver installations that work with the Windows PnP detection, prompt, and install sequence? Most vendors tell you to avoid that by first auto-running their installation CD or running their Setup.exe, and then plugging in the USB device only after the "drivers" are loaded.

One reason may be because the vendor wants to install a range of software that is broader than that handled by the device driver installation purpose. And this is where the meta-bug comes in, because whereas drivers are suppressed when the device is not found, the rest of the bundled software is not. This is what causes your Display Properties dialogs to crash when you replace your graphics card, if the old graphics card left device-specific code hooked into the Properties page.

In this case, the meta-bug caused an apparent inability to uninstall Intel Audio Studio. With the new motherboard in place, a dialog appears on every boot to the effect that Intel Audio Studio doesn't work with the system's (different) audio, prompting my attempts to uninstall the software via Add/Remove Programs.

What is particularly annoying, is that Intel's web site has nothing I could find on this issue - either via their own site search, or combining www.intel.com with appropriate search terms in a general Internet Google search.

Web Forum Login Blues

2008-04-11T01:58:00.001-07:00

Technorati tags: Forums, Web

One of the challenges in web design and web forums in particular, is how to handle the requirement to log in for more functional access.

Whenever a login prompt pops up, folks will often simply leave, rather than log in, even if they are already registered at the site. So immediate login isn't usually what you want to do with web forums; for one thing, you will lose folks who find your forums via search results that jump into forum content.

So web forums generally let you browse around until you want to do something that requires you to login, such as start new threads, reply to posts, etc.

Even at this point, I'm still inclined to give up rather than continue, when I see a login prompt. Why? Because I anticipate a hassle of not only login in, which I can handle, but having to re-navigate my way back to what I was doing - i.e. which forum, which thread, which one of 27 pages of posts (sometimes requiring 22 "next" because there's no "go to end" where the most recent stuff is found) and which post on that page, then the edit box I may have already started.

The Bart forums hosted here get it right; you sign in, and you are returned exactly to context you left before you had to log in.

Microsoft's web reader for their newsgroups gets it wrong, at least in the context of linking to their forums via other Microsoft sites. Here's the repro; start here...

http://www.microsoft.com/windows/ie/ie8/welcome/en/default.html

...and then at the bottom right of the page, right-click, Open In New Tab (the method I used, a straight click will likely do the same) on "Community Forums", taking you here:

http://www.microsoft.com/communities/newsgroups/list/en-us/default.aspx?dg=microsoft.public.internetexplorer.beta&cat=en_us_2BAF8EC5-645C-4477-A380-0F1CF6C102F9&lang=en&cr=us

So far, so good; we're in blah.blah.blah.ie8.blah which is where we want to be, and we start reading, then decide we want to post a new thread. Oops, now we want to login, so we do, and when we get back, we are no longer in the context of the newsgroup we wanted to visit. We didn't note the name of that newsgroup because we thought we'd always be able to link into it directly from the IE8 beta page.

Now I know Microsoft are battling to win over many of us usenet die-hards to embrace the web UI to their newsgroups. This is IMO a serious stumbling block, not only for us, but also for folks discovering newsgroups for the first time, without usenet preconceptions and expectations.

The above is as tested on XP SP2 with IE8 beta 1 installed over IE7, IE8 Standards Mode, set to prompt on active content, all such prompts OK''d.

Workaround 1: Remember what the newsgroup is called, duh! Not so easy in that it's not content that can be cut and pasted, in terms of the UI, and the nature of the data is a pain to remember. Breaks the concept of "let the PC do it", too, plus that only gets you back as far as the start of the newsgroup, before you navigated within it.

Workaround 2: Log in, then go back to the IE8 page and repeat the link to the forums (which Microsoft may also refer to as "communities"). This time you will navigate into a logged-in state, and all will be well, though any navigation you did within the forum would have to be re-done. For best practice (i.e. shouldn't be required, but...) remember to log out of both forum tabs when done.

NTFS vs. FATxx Data Recovery

2008-03-30T16:24:00.001-07:00

Technorati tags: Data recovery

By now, I've racked up some mileage with data recovery in FATxx and NTFS, using R-Studio (paid), GetDataBack (demo), manually via ye olde Norton DiskEdit (paid), and free Restoration, File Recovery 4 and Handy Recovery, and a pattern emerges.

Dispelling some myths

NTFS has features that allow transactions to be reversed, and there's much talk of how it "preserves data" in the face of corruption. But all it really preserves is the sanity of the file system and metadata; your actual file contents are not included in these schemes of things.

Further, measures such as the above, plus automated file system repair after bad exits from Windows, are geared to the interruption of sane file system activity. They can do nothing to minimize the impact of insane file system activity, as happens when bad RAM corrupts addresses and contents of sector writes, nor can they ameliorate the impact of bad sectors encountered on reads (when the data is not in memory to write somewhere else, it can only be lost).

From an OS vendors' perspective, there's no reason to consider it a failing to not be able to handle bad RAM and bad sectors; after all, it's not the OS vendor's responsibility to work properly under these conditions. But they occur in the real world, and from a user's perspective, it's best if they are handled as well as possible.

The best defences against this sort of corruption is redundancy of critical information, such as the duplication of FATs, or less obviously, the ability to deduce one set of metadata from another set of cues. Comparison of these redundant metadata allows the integrity of the file system to be checked, and anomalies detected.

Random sector loss

Loss of sector contents to corruption or physical disk defects is often not random, but weighted towards those parts of the disk that are accessed (bad sectors) or written (bad sectors and corruption) the most often. This enlarges the importance of critical parts of the file system that do not change location and that are often accessed.

When this happens, there are generally three levels of recovery.

The first level, and easiest, is to simply copy off the files that are not corrupted. Before doing so, you have to exclude bad hardware that can corrupt the process (e.g. bad RAM), and then you make a beeline for your most important files, copying them off the stricken hard drive - even before you surface scan the drive to see if there are in fact failing sectors on it, or attempt a full partition image copy. This way, you get at least some data off even if the hard drive has less than an hour before dying completely.

You may find some locations can't be copied for various reasons that break down to invalid file system structure, physical bad sectors, overwritten contents, or cleanly missing files that have been erased. If the hard drive is physically bad, you'd then attempt a partition copy to a known-good drive. If you want to recover cleanly erased files, or attempt correction of corrupted file systems, then this partition copy must include everything in that space, rather than just the files as defined by the existing file system.

The second level of recovery is where you regain access to lost files by repairing the file system's logic and structure. This includes finding partitions and rebuilding partition tables, finding lost directory trees and rebuilding missing root directories, repairing mismatched FATs and so on. I generally do this manually for FATxx, whereas tools like R-Studio, GetDataBack etc. attempt to automate the process for both FATxx and NTFS.

In the case of FATxx, the most common requirements are to rebuild a lost root directory by creating scratch entries pointing to all discovered directories that have .. (i.e. root) as their parent, and to build a matched and valid pair of FATs by selectively coping sectors from one FAT to the other.

Recovered data is often in perfect condition, but may be corrupted if file system cues are incomplete, or if material was overwritten. Bad sectors announce their presence, but if bad RAM had corrupted the contents of what was written to disk, then these files will pass file system structural checks, yet contain corrupted data.

The third level of logical data recovery is the most desperate, with the poorest results. This is where you have lost file system structural cues to cluster chaining and/or the directory entries that describe the files.

Where cluster chaining information is lost, one generally assumes sequential order of clusters (i.e. no fragmentation) terminated by the start of other files or directories, as cued by found directories and the start cluster addresses defined by the entries within these. In the case of FATxx, I generally chain the entire volume as one contiguous cross-linked file by pasting "flat FATs" into place. Files can be copied off a la first level recovery once this is done, but no file system writes should be allowed.

If directory entries are lost, then the start of files and directories can be detected by cues within the missing material itself. Subdirectories in FATxx start with . and .. entries defining self and parent, respectively, and these are the cues that "search for directories" generally use in DiskEdit and others. Many file types contain known header bytes and known offsets (e.g. MZ for Windows code files) and this is used to recover "files" from raw disk by Handy Recovery and others - a particularly useful tactic for recovering photos from camera storage, especially if the size is typical and known.

Results

I have found that when a FATxx volume suffers bad sectors, it is typical to lose 5M to 50M material from a file set ranging from 20G to 200G in size. The remainder is generally perfectly recovered, and most recovery is level one stuff, complicated only by the need to step over "disk error" messages and retry bog-downs.

When level two recovery is needed, the results are often as good as the above, but the risks of corrupted contents within recovered files are higher. The risk is higher if bad RAM has been a factor, and is particularly high if a "flat FAT" has to be assumed.

In contrast, when I use R-Studio and similar tools to recover files from NTFS volumes with similar damage, I typically get a very small directory tree that contains little that is useful. Invariably I have to use level three methods to find the data I want. Instead of getting 95% of files back in good (if not perfect) condition, I'll typically lose 95%, and the 5% I get is typically not what I am looking for anyway.

Level three recovery is generally a mess. Flat-FAT assumptions ensure multi-cluster files are often corrupted, and loss of meaningful file names, directory paths and actual file lengths often make it hard to interpret and use the recovered files (or "files").

Why does mild corruption of FATxx typically return 90%+ of material in good condition, whereas NTFS typically returns garbage? It appears is if the directory information is particularly easy to lose in NTFS. I don't believe all the tools I've used, are unable to match the manual logic I use when repairing FATxx file systems via DeskEdit.

Survivability strategies

Sure, backups are the best way to mitigate future risks of data loss, but realistically, folks ask for data recovery so often that one should look beyond that, and set up file systems and hard drive volumes with an eye to survivability and recovery.

Data corruption occurs during disk writes, and there may be a relationship between access and bad sectors. So the first strategy is to keep your data where there is less disk write activity, and disk access in general. That means separating the OS partition, with its busy temp, swap and web cache writes, from the data you wish to survive.

At this point, you have opposing requirements. For performance, you'd want to locate the data volume close to the system partition, but survivability would be best if it was far way, where the heads seldom go. The solution to this is to locate the data close, and automate a daily unattended backup that zips the data set into archives kept on a volume at the far end of the hard drive, keeping the last few of these on a FIFO basis.

One strategy to simplify data recovery is to use a small volume to contain only your most important files. That means level three recovery has less chaff to wade through (consider picking out your 1 000 photos from 100 000 web cache pictures in the same mass of recovered nnnnn.JPG files), and you can peel off the whole volume as a manageable slab of raw sectors to paste onto a known-good hard drive for recovery while the rest of the system goes back to work in the field.

The loss of cluster chaining information means that any file longer than one cluster may contain garbage. FATxx stores this chaining information within the FATs, which also cue which clusters are unused, which are bad, and which terminate data cluster chains. NTFS stores this information more compactly; cluster runs are stored as start, length value pairs, whereas a single bitmap holds the used/free status of all data clusters, somewhat like a 1-bit FAT.

Either way, this chaining information is frequently written and may not move on the disk, and both of these factors increase the risk of loss. A strategy to mitigate this common scenario is to deliberately favour large cluster size for small yet crucial files, so that ideally, all data is held in the first and only data cluster. This is why I still often use FAT16, rather than FAT32, for small data volumes holding small files.

Another strategy is to avoid storing material in the root directory itself (for some reason, this is often trashed, especially by some malware payloads on C:) and to also avoid long and deeply-nested paths. Some recovery methods, e.g. using ReadNTFS on a stricken NTFS volume, requires you to navigate through each step of a long path, which is tedious due to ReadNTFS's slowness, the need to step over bad sector retries along the way, and the risks of the path being broken by a trashed directory along the way.

Some recovery tools (including anything DOS-based, such as DiskEdit and ReadNTFS)can't be safely used beyond the 137G line, so it is best to keep crucial material within this limit. Because ReadNTFS is one of the only tools that accesses NTFS files independently of the NTFS.sys driver, it may be the only way into to NTFS volumes corrupted in ways that crash NTFS.sys!

Given the poor results I see when recovering data from NTFS, I'd have to recommend using FATxx rather than NTFS as a data survivability strategy. If readers can attain better results with other recovery tools for NTFS, then please describe your mileage with these in the comments section!

Why "One Bad Sector" Often Kills You

2008-03-27T16:43:00.000-07:00

Technorati tags: Data Recovery

Has it ever seemed to you, that if there's "one bad sector" on a hard drive, it will often be where it can hurt you the most?

Well, there may be reasons for that - and the take-home should affect the way file systems such as NTFS are designed.

As it is, when I see early bad sectors, they are often in frequently-accessed locations. This isn't because I don't look for bad sectors unless the PC fails, as I routinely do surface scans whenever PCs come in for any sort of work. It's good CYA practice to do this, saving you from making excuses when what you were asked to do, causes damage due to unexpected pre-existing hardware damage.

Why might frequently-accessed sectors fail?

You could postulate physical wear of the disk surface, especially if the air space is polluted with particular matter, e.g. from a failed filter or seal, or debris thrown up from a head strike. This might wear the disk surface most, wherever the heads were most often positioned.

You could postulate higher write traffic to increase the risk of a poor or failed write that invalidates the sector.

Or you could note that if a head crash is going to happen, it's most likely to happen where the heads are most often positioned.

All of the above is worse if the frequently-accessed material is never relocated by file updates, or defrag. That may apply to files that are always "in use", as well as structural elements of the file system such as FATs, NTFS MFT, etc.

Core code files may also be candidates if they have to be repeatedly re-read after being paged out of RAM - suggesting a risk mechanism that involves access rather than writes, if so.

As it is, I've often seen "one bad sector" within a crucial registry hive, or one of the core code files back in the Win9x days. Both of these cause particular failure patterns that I've seen often enough to recognize, e.g. the Win9x system that rolls smoothly from boot to desktop and directly to shutdown, with no error messages, that happens when one of the core code files is bent.

I've often seen "one bad sector" within frequently-updated file system elements, such as FATs, NTFS "used sectors" bitmap, root directory, etc. which may explain why data recovery from bad-sector-stricken NTFS is so often unsatisfactory.

But that's another post...

Google Desktop vs. Vista Search

2008-03-24T03:46:00.001-07:00

Technorati tags: Microsoft, Apple, Safari, Google, anti-trust

Google accuses Microsoft of anti-competitive behaviour, in that Vista currently leverages its own desktop search over Google Desktop and other alternatives. This issue is well-covered elsewhere, but some thoughts come to mind...

Isn't Google hardwired as the search engine within Apple's Safari?

Isn't Apple pushing Safari via the "software update" process as bundled with iTunes and QuickTime, even if the user didn't have Safari installed to begin with?

I'm seeing a lot of black pots and kettles here.

More to the point: If an alternate serach is chosen by the user or system builder, is the built-in Microsoft indexer stripped out? This article suggests it won't be.

That's the ball to watch, because so far, Microsoft's approach to enabling competing subsystems has been to redirect UI to point to the 3rd-party replacement, without removing the integrated Microsoft alternative.

That means the code bloat and exploitability risks of the Microsoft stuff remains, and that in turn makes it impossible for competitors to reduce the overall "cost" of that functionality (as using something else still incurs the "cost" of the Microsoft subsystem as well).

This is particularly onerous when the Microsoft subsystem is still running underfoot.

For an example of the sort of problems that can arise; if you have an edition of Vista that does not offer the "Previous Versions" feature, you still have that code running underfoot, maintaining previous versions of your data files. If someone subsequently upgrades Vista to an edition that does include "Previous Versions", then they can recover "previous versions" of your data files, even though those files were altered before Vista was upgraded.

So it's not enough to give Google (and presumably others, this complaint is not just for the benefit of the search king, is it?) equal or pre-eminant UI space. If one has to accept the runtime overhead of some 3rd-party's indexer, then it's imperitive that Microsoft's indexer is not left running as well.

As it is, indexer overhead is a big performance complaint with Vista. If 3rd-party desktop search has to suffer the overhead of two different indexers, the dice are still loaded against the competition, because no matter how much more efficient the 3rd-party indexer may be, the overall result is worse performance.

SysClean on Bart PE

2008-02-21T03:29:00.001-08:00

Technorati tags: Bart, antivirus, mOS, SysClean

Trend SysClean is a self-contained, stand-alone malware cleaner that can be used from the Bart PE boot CDR environment. Unlike many free cleaners such as McAfee Stinger and Avast Cleaner, it detects most things that a full-range resident av would detect, rather than a small subset of these.

On the face of it, it should be easy to run SysClean from a Bart CDR boot, but there are a few gotchas that can mess you up. If you're one who has sorted those out ages ago, yet recent found SysClean to no longer work in Bart.

Either way, read on...

How SysClean works

SysClean exists as a SysClean.com engine, plus a larger signature data file with names such as LPT%VPN.xxx, where xxx is a 3-digit number that rolls over from .999 to .000 or .001 whenever there have been that many updates. You can wildcard the data file as LPT$*.*, LPT$VPN.*, LPT$*.???, etc.

SysClean does not have to be installed before use, which makes it attractive as an intervention scanner. There is no integrated updater, so you'd manually download the latest signature data before use. As the engine is also subject to change, I'd recommend downloading a fresh engine alone with new signature data.

SysClean.com is not a true .com file, i.e. it is not a DOS-era 16-bit memory image of code that runs with all segment registers set to the same 64k space. Instead, it is a Win32 executable that unpacks itself and then jumps into itself to run.

From all of the above, you can predict pitfalls when using SysClean from Bart CDR.

General issues

The easy way to avoid these pitfalls, is to copy the files to a HD location and run them from there, all done within the same Bart boot session.

If you want to integrate via a SysClean plugin into Bart, you have to essentially automate this process, as well as avoiding a few other issues.

As SysClean writes to its own location, that location must be writable (i.e. can't run directly off the CDR) and must have enough free space to unpack (which may not be the case if running within a small RAM drive).

As SysClean.com chains into the SysClean.exe that it spawns, you must ensure your automation logic does not prematurely continue, i.e. after SysClean.com terminates but while SysClean.exe is still running. The Start /W approach is likely to fail in this way.

SysClean launches sub-tasks, and that means it may fail in environments that impose limits on the number of tasks that can run at the same time. WinPE, Bart PE and Windows XP Starter Edition may fall into this category. If you have one "wizard" batch file that launches another batch file that launches a set of scanners in sequence, then scanners that start additional processes may hit the limit.

Some of the scans are launched as sub-tasks that run in "DOS-style" CLI windows. If you used a tool within your launcher batch file to hide the batch file window, this may hide the CLI subtasks as well - creating the impression these aren't running from Bart. I haven't gone into this, e.g. removed my CLI winder hider etc. and thus am not sure the CLI scans are done in Bart, or really are skipped.

Recent failure pattern

If you've beaten all of the above problems years ago, you may have hit the following failure pattern recently...

SysClean extracts itself and runs OK, presenting you with its GUI. You then slick the Scan button and it starts scanning memory, before scanning files. But it never completes this process; even after the CD and hard drive burbling stops, it just sits there "scanning memory..." forever.

The system and app haven't crashed. If you click to stop the scan, nothing happens, but if you click the [x] to close SysClean's window, that works.

Quick fix

If you run SysClean again, within the same Bart session, it works perfectly!

Why does if fail the first time?

A few months ago, SysClean changed its behaviour; at the start of the scanning process, between checking memory and scanning files, it now pops up an "OK" status dialog, to the effect that no viruses were found in memory.

When run in Bart, this dialog never appears - so you can't see it and you can't click it away. And thus, SysClean will stall, waiting for an "OK" click that will never come.

Why does it work the second time?

When SysClean runs, it spawns a resident process called TSC.BIN that remains running after SysClean is done. This is spawned before the failed "OK" prompt; I suspect it's spawned as early as possible, to run as "air cover" should any active malware code try to interfere with the scanning process.

The problematic prompt is only launched if TSC.BIN is not already running when SysClean starts its scan (perhaps TSC.BIN is itself the origin of the prompt, as part of its initialisation).

So the first scan starts TSC.BIN and suffers the UI stall, whereas all subsequent scans during the same Bart session will already have TSC.BIN running and are OK.

I see one SysClean plugin approach may side-step this issue by scooping the extracted files into the plugin, rather than having the plugin run SysClean.com to extract them at runtime. This may avoid the problem if it is the extraction process that triggers the dialog - though that seems unlikely.

Person Of The Year: The Unknown Assailant

2008-01-27T01:53:00.001-08:00

Around about now, Time magazine usually has a "person of the year" award. One year they got cute with this, e.g. citing "you" as that person, empowered as you are by your access to online content creation, etc.

But consider the extent to which your range of activities are constrained by the possible actions of anonymous entities. Some of us are well accustomed to that in "real life", and now it is carving deep inroads into the online experience.

Several business practices have had to be abandoned within years of becoming possible, such as vendor-to-consumer email, online greeting cards, etc. For example, savvy users won't click on attachments or links within email "from" vendors, which leaves the dumbo demographic for such practices. That may no longer be where the smart money can be earned, but it remains a good place to snatch dumb money - so these practices become dominated by malicious, value-free "vendors".

And so we see whole chunks of Internet practice and OS "features" being abandoned to my nomination for "person" of the year; the unknown assailant.

What You Can't See

2008-01-03T03:49:00.001-08:00

Last weekend I was at the Festival of Chariots, and got talking to someone there. After asking what I did, she said "ah, you have a scientific brain; you don't believe in what you can't see".

That's not the case at all; if anything, science reminds us of the limitations of what we can see, and scientific rigor suggests we can't make authoritative claims on what we can't see - including claims as to whether or not such things exist.

In fact, only one of our senses works off-planet, and that is sensitive to a very narrow spectrum of photon energies. Much of what we currently "see", is visualized via artificial methods, i.e. sensory equipment and arbitrary interpretive processing.

Finding Recent Comments

2008-01-03T01:01:00.001-08:00

This is one reason I prefer WordPress to Blogger. Once I've OK'd comments, I have to smell where they are if I want to reply to them with comments on my own - there's no UI to access "recent comments" irrespective of which post they are commenting.

Post-Google Blogger being what it is, there's also no easy route to submit this feedback to the blogging service (no, I don't want to do the whole "forum" thing, thanks). Live Spaces is better there; they're always keen for feedback and new ideas.

Malware "War", Lost Territory

2007-12-17T05:14:00.001-08:00

Technorati tags: Malware, Safety

I've often seen the malware situation described as a "war", and conventionally, wars are fought over territory.

What territory has been lost to malware?

Consider various integration points that are now routinely defended against usage, on the basis that the only things likely to use these, are malware. These OS "features" are now effectively "owned" by malware, in that legitimate software will trigger defence alerts if they are used.

Consider a number of ill-advised features that are designed to allow arbitrary material to automate the system, e.g. MS Word auto-running macros, auto-running scripts in HTML email "messages", \Autorun.inf processing on USB flash drives, etc. Today, these will typically be disabled, because the most likely use will be by malware. So Malware "own" that, too.

Consider several business models that involve messages, attachments or links sent by the service's site, such as email greeting cards. As malware can arrive via forgeries of such messages, usage is limited to those who are too dumb to know the risk they are expecting the recipient to take, which is a smaller and more limited demographic than when such services were first started. Effectively, these kinds of businesses and practices have been killed by malware.

Should we scorch and abandon some of this territory? For example, remove OS integration points that are hardly ever used by anything other than malware?

Should we assess likely future "ownership" before creating new technologies and features that are likely to be swamped by malware?

Norton Security Scan - False Positives

2007-11-18T04:15:00.001-08:00

Technorati tags: Malware, Risk Management, Safety, Bug

The Norton Security Scan utility is free, and bundled with the Google Pack. It's an on-demand scanner that looks for malware and risks.

Unfortunately, it detects protective settings applied by Spyware Blaster and similar tools, as being the malware these tools are protecting against. This is a generic type of bug that often arises when tools assume anything other than default is a hostile change, or when overly-loose detection cues are in effect.

Specifically, settings within HKCU's P3P\History that block unwanted cookies, are detected as evidence of malware. In the case I'm currently working on, only around 5 of over 100 protective entries were detected in this way.

The tool then claims it is unable to fix these problems, which is just as well, as doing so would actually weaken system safety. The end result is similar to Winfixer et al, i.e. false-positive (actually, reverse-positive) detections plus referral to feeware products if these are to be "fixed" - so one hopes Symantec will fix this sooner rather than later.

The case I'm working on is interesting, as it was brought in because it was slowing down, with malware as the suspected cause. Formal scanning finds no active malware, and one wonders if the slowdown was the result of installing Google desktop etc., with the false-positive from Norton Security Scan as the red herring.

I'd like to try Norton Security Scan within mOS contexts such as Bart or WinPE CDR boot, but it appears as if the product is available only via Google Pack. There are no references to it at Symantec's site, and the FAQ doesn't seem to consider "so where can I download this thing?" to be "frequently asked".

SARS Tax Returns vs. Acrobat Reader

2007-11-18T02:52:00.001-08:00

Technorati tags: South Africa, Risk Management, Safety

Those using the South African Revenue Service (SARS) e-filing facility may find non-default safety settings within Adobe Acrobat Reader get in the way.

How to fix

Most likely you need only enable JavaScript, but when I had to troubleshoot this in the field, I applied all of the following settings...

Run Adobe Acrobat Reader 8.x

Edit menu, Preferences

JavaScript icon, [x] Enable Acrobat JavaScript

Multimedia Trust icon, Trusted Documents radio button, [x] Allow Multimedia Operations

Multimedia Trust icon, Other Documents radio button, [x] Allow Multimedia Operations

Trust Manager icon, [x] Allow Opening of Non-PDF File Attachments with External Applications

...and reversed them for safety when done.

Why use safer settings?

If these non-default settings stop things like SARS e-filing from working, why apply them?

Because Acrobat files are already being exploited by spam, and a significant safety gap exists between what you think a .PDF is (i.e. a data format that is safe to read) and what it can do (automate your system via JavaScript, launching of other files and code, etc.).

Acrobat Reader is an exploitable surface that has often been patched to "fix" it, and for which unpatched vulnerabilities often exist. Commercial enterprises have already exploited the by-design safety gap, e.g. by having .PDF documents "call home" when they are read, so that their usage can be tracked.

So one should keep Acrobat Reader on a very short leash, or use something else to "open" .PDF and other Acrobat file types.

Understanding Integers

2007-10-12T18:07:00.001-07:00

Technorati tags: Number theory

Which is the largest of these rational numbers?

1.5075643002
5.2
5.193
5.213454

If you say 5.213454, then you're still thinking in integer terms. If you say 1.5075643002 is the largest within a rational number frame of reference, then you thinking as I am right now.

Is Pi an irrational number, or have we just not effectively defined it yet? With the Halting Problem in mind, can we ever determine whether Pi is rational or irrational? On that basis, is there such a thing as irrational real numbers, or are these just rational numbers beyond the reach of our precision? Unlike most, this last question will be answered within this article.

Rational numbers

I was taught that rational numbers were those that can be expressed as one integer divided by another - but I'm reconsidering that as numbers that lie between fixed bounds, or more intuitively, "parts of a whole".

When we deal with rational numbers in everyday life, we're not really dealing with rational numbers as I conceptualize them within this article. We're just dealing with clumps of integers.

To enumerate things, there needs to be a frame of reference. If you say "six" I'll ask 'six what?', and if you say "a quarter", I'll ask 'a quarter of what?'

Usually, the answer is something quite arbitrary, such as "king's toenails". Want less arbitrary? How about "the length of a certain platinum-iridium bar in Paris stored at a particular temperature" - feel better now?

Your cutting machine won't explode in a cloud of quantum dust if you set it to a fraction of a millimeter; within the machine's "integers", are just more smaller "integers". If you think you understand anything about rational numbers from contexts like these, you're kidding yourself, in my humble opinion.

Integers

To put teeth into integers, they have to enumerate something fundamental, something atomic. By atomic, we used to say "something that cannot be divided further"; today we might say "something that cannot be divided further without applying a state change, or dropping down into a deeper level of abstraction".

Ah - now we're getting somewhere! Levels of abstraction, total information content, dimensions of an array... think of chemistry as a level of abstraction "above" nuclear physics, or the computer's digital level of abstraction as "above" that of analog volts, nanometers and nanoseconds.

If layers of abstraction are properly nested (are they?), then each may appear to be a closed single atom from "above", rational numbers from "within", and an infinite series of integers from "below". Or not - toss that around your skull for a while, and see what you conclude.

Closed systems

Within a closed system, there may be a large but finite number of atomic values (or integers, in the non-ordered sense), being the total information content of that system. If rational numbers are used to describe entities within the system, they are by necessity defined as values between 0 and 1, where 1 = "the system". In this sense, 7.45 is not a rational number, but might be considered as an offset 7 from outside the system, and .45 within the system.

You might consider "size" as solidity of existence, i.e. the precision (space) or certainty (time, or probability) at which an entity is defined. If you can define it right down to specifying exactly which "atom" it is, you have reached the maximum information for the closed system. So 0.45765432 is a "larger" number than 0.5, in terms of this closed-system logic.

You can consider integers vs. rational numbers as being defined by whether you are specifying (or resolving) things in a closed system (rational numbers, as described within this article) or ordering things in an open system (integers).

What closes an integer system is your confidence in the order of the integers you enumerate. What closes a rational system is whether you can "see" right down to the underlying "atoms".

Information and energy

Can one specify an entity within a closed system with a precision so high that it is absolute, within the context of that system?

We may generalize Pauli's exclusion principle to state that no two entities may be identical in all respects (or rather, that if they were, they would define the same entity).

Then there's Heisenberg's uncertainty principle, that predicts an inability to determine all information about an entity, without instantly invalidating that information. Instantly? For a zero value of time to exist, implies an "atom" of time that zero time is devoid of... otherwise that "zero" is just a probability smudge near one end of some unsigned axis (or an arbitrary "mid-"point of a signed axis).

Can you fix (say) an electron so that its state is identical for a certain period of time after it is observed? How much energy is required to do that? Intuitively, I see a relationship between specificity, i.e. the precision or certainty to which an entity is defined, and the energy required to maintain that state.

Entropy

If "things fall apart", then why? Where does the automatic blurring of information come from? Why does it take more work to create a piece of metal that is 2.6578135g in mass than one manufactured to 2.65g with a tolerance of 0.005g?

One answer may be; from deeper abstraction layers nested within what the current abstraction layer sees as being integer, or "atomic". The nuclear climate may affect where an electron currently "is" and how likely it is to change energy state; what appears to be a static chemical equilibrium could "spontaneously" change, just as what appears to be reliable digital processing can be corrupted by analog voltage changes that exceed the trigger points that define the digital layer of abstraction.

In this sense, the arrangement of sub-nuclear entities may define whether something is a neutron or a proton with an electron somewhere out there; the difference is profound for the chemical layer of abstraction above.

To freeze a state within a given layer of abstraction, may require mastery over deeper levels of abstraction that may "randomize" it.

Existence

What does it mean, to exist? One can sense this as the application of specificity, or a stipulation of information that defines what then exists. Our perspective is that mass really exists, and just happens to be massive in terms of the energy (information?) contained within it.

There's a sense of energy-information conservation in reactions such as matter and antimatter annihilating their mass and producing a large amount of energy. How much energy? Does that imply the magnitude of information that defined the masses, or mass and anti-mass? Do you like your integers signed or unsigned? Is the difference merely a matter of externalizing one piece of information as the "sign bit"? What do things look like if you externalize two bits in that way?

Like most of my head-spin articles, this one leaves you hanging at this point. No tidy summary of what I "told" you, as I have no certainty on any of this; think of this article as a question (or RFC, if you like), not a statement.

Navigation via Recent Comments

2007-10-10T10:15:00.000-07:00

Here's something this blog host needs; an ability to zoom in on the most recent comments, irrespective of where they are, so one can comment on them. If this facility is present, it needs to be more discoverable.

As it is, one goes in and moderates unmoderated comments, but having done so, they vanish from easy navigation so one can't follow them up to reply.

Oh... some more general "CQspace" news; I intend to focus more on maintenance OS issues and development (with a small "d", i.e. how to make your own projects by tailoring existing mOSs) and will do that at what is currently called "CQuirke's Linux Curve", as that blog host appears to have the best oomph to carry the blog-to-website transition I am after. As part of that focus, I'll still be learning Linux and blogging that as I go, but it will be a subset of that site as a whole.

New Blog Elsewhere

2007-09-11T16:35:00.001-07:00

Technorati tags: Blogging

I've started a third blog here, mainly because I liked the look of the hosting service:

No "bad cookie" alerts, unlike here
Richer feature set
Better suited to "normal" web site structure

Normally, each blog has a "theme"; this one is general, the other blog is about Vista, and the new one might be about Linux if I get traction with that.

I've checked out Linux from time to time, and this time I'm prompted to do so by what I see as deteriorating vendor trustworthiness, coupled with tighter vendor dependence; activation false-positives, WGA service failures that triggered (in this case, mild) DoS effects, OEM MS Office 2007 sold as "air boxes" i.e. no installation disks, and poor responsiveness and documentation on these issues.

I'm also checking out Linux as a potential maintenance OS (mOS) for Vista; possibly one that can service all Windows versions plus Linux itself. The newest Ubuntu 7.x claims safe writeable support for NTFS, and until we see RunScanner functionality for Vista, that evens the playing field compared to Bart and WinPE (in other words, none of them can do for Vista what Bart can do for XP).

I expect it will take a year to build satisfactory skills in mOS for Vista, and longer to get a handle on Linux - which means if I want to be positioned to switch to Linux in a few year's time, the time to start studying it is now.

The standards I set for myself as a PC builder require custom-installable disks to ship for all installed software. Failing that, unrestricted and anonymous download is an acceptable alternative only if that is compatible with systems that have either no Internet connectivity, or slow and costly dial-up access.

OEM MS Office 2007 already fails this standard, and I refuse to sell it accordingly. Given the stealth with which Microsoft has manipulated OEM MS Office 2007, I cannot assume similar changes impacting on Vista will occur only when the next version of Windows is released. So starting on a years-long mOS development path may be a waste of time, if such work is applicable to Windows alone.

The first prize would be a Windows that isn't chained to sucky vendor politics, and I will continue to work towards that where possible. If Windows becomes unacceptable, it would be quite a setback in many ways, but that lump may have to be swallowed... let's hope cooler heads kick in an Microsoft, so that we can still stay with the platform we already know and use!

WGA, Product Activation, Kafka

2007-09-07T01:07:00.001-07:00

Technorati tags: WGA, Activation, Vandor, Consumer Rights, Surrealism

If Kafka wrote the Windows activation/WGA FAQ...

A: You have been found guilty and have been sentenced to die in 3 days. Would you like to appeal?

Q: What crime am I being charged with?

A: Our code has found you guilty of being guilty in the opinion of our code. You have already been found guilty and sentenced. Would you like to appeal?

Q: What laws have I broken?

A: The laws you have broken are those we assigned ourselves via the EUL"A" you consented to when you accepted our product.

Q: I don't remember discussions about an End User License Agreement?

A: Well, you wouldn't; we find it more effective to just write that up ourselves.

Q: I need details... what exact laws have I broken?

A: We find it more effective not to disclose the details on how our code investigates such matters, or what criteria are used to determine the breaking of our laws. All you need to know is that you have been found guilty and sentenced.

Would you like to appeal?

Q: What do you mean "die in 3 days"?

A: In three day's time, your heart will be removed and further processing will not be possible. If you do not have recourse to another heart and/or cardiac troubleshooting skills, you will remain inert. Your body parts will still be available to those with the skills to access them; don't worry, no personal data will be lost, though of course you will need a new heart from us to work with that data again.

Would you like to appeal?

Q: OK, I'd like to appeal. Who do I appeal to?

A: Us, of course. Phone the number, answer a few trick questions like "press 1 if you have two or more, press 2 if you have only one" etc. and then ask to speak to a human. Convince the human you are innocent and your death sentence will be set aside. If you are innocent, you have nothing to fear!

Q: What do you mean "if I'm innocent"? You've just told me you've found me guilty, and refused to tell me exactly of what I'm guilty?

A: This is true, but we are not unfair. You do have the right to appeal, as at September 2007.

Q: So how do I present my case?

A: Leave that to us. We will ask you questions, and based on your answers, we will decide if perhaps we arrested, tried and sentenced you in error, or whether our code works as designed.

Q: Works as designed? What is it designed to do?

A: It's designed to determine whether you are guilty or not. We find it is more effective not to disclose details of how it does this.

Q: Can I review the evidence?

A: We find it is more effective if the guilty party is not permitted to review the evidence, and thus we provide no tools to do so, nor do we provide documentation of what this evidence may be. Any such documentation you may find will have been subject to change. We will not tell you if, when or how it has changed, if indeed it has. We find it is more effective this way.

Q: OK! Hey, everything's fine! I spoke to the human and explained what happened (which was easy in my case; nothing happened or changed, you just charged me out of the blue) and they set aside my sentence! Thanks you running such a wonderful system that allows a lowly wretch like me to live again!!

A: It's a pleasure, glad to help ;-)

Three Little Pigs Build Computers

2007-09-02T01:45:00.001-07:00

Technorati tags: Safety, Parable

If you don't like long fairy tales, skip ahead to the conclusions!

Once upon a time, there were three little pigs who set out to become building contractors.

One insisted everyone build their house on his land using only his materials, and charged too much money. He didn't sell that many houses, and this story is not about that little pig.

One believed that people should build their own houses, and that houses should be built for free. Many people were very interested in this, and often started building such houses, but found it too difficult and gave up, and this story is not about that little pig either.

Pig Makes Good: The Early Years

All of these pigs grew up in houses made of bricks, but this was a new planet where bricks weren't available (I did say "set out", didn't I?), so they had to make do with other materials instead. At first, they made houses out of these materials the way their brick houses were made back home, but there were so many people wanting houses in the same place that they started joining them together in various ways.

The main pig became very successful, not only making houses for nearly everyone on the planet, but employing lots of builders to do so; soon, there wasn't a single builder who knew everything needed to build a complete house.

As more people came to the new planet, most of the big's earnings came from building hotels and blocks of flats. They still built lots of houses, but stopped thinking about how those would be made because that wasn't where the money was, and besides, those folks will always buy their houses anyway.

Wolf Atrocities: The Response

After a while, folks started complaining that homeowners were being eaten by wolves, and the quality of houses came into question. Wolves will be wolves, it was agreed, but surely the idea of a house is to protect one from them?

Some folks suggested building houses out of sticks instead of straw, but the pig said "we have too many pre-built walls that we already made out of straw; it would take far too long to re-do everything in sticks".

Others felt that building out of straw vs. sticks didn't matter too much, as long as you did something about the open windows and weak door hinges.

The pig said "if you want stronger doors, speak to the Door Lock team. What's that about 'hinges'? We don't have a "Hinge Team", so we can't pass those suggestions anywhere. I promise we'll make doors with even stronger locks in future! (the squeaky things at the other end of the door will stay the same, of course)"

The pig also said "we've always built with open windows, and other businesses have come to depend on them. How are folks going to deliver goods and services if they can't climb in through the window? Why not retire to one of the bedrooms, and lock yourself in? If you get eaten, it's your fault, because you will insist in walking around in other rooms that aren't meant for you - you are a resident, not a chef or a barman, so you don't belong in the kitchen or living room."

So the new houses were built with stronger locks and new bedroom doors. And as folks still needed to eat and go to the bathroom, they'd leave the bedroom doors unlocked and get eaten while in the other rooms.

Conclusions

1. The past can tell you only so much about the future.

If you focus on large-volume quality data from the real world while designing new products, you will design products that solve yesterday's problems while being wide open to tomorrow's new problems.

To avoid this trap, you need to brainstorm new designs with homeowners from the start, rather than present them with a near-completed beta product where the design is already cast in stone. You also need to listen to theorists who cannot point to detailed real-world data because what they are talking about does not yet exist in the real world, and pay as much attention to these as you do to the detailed real-world feedback you get on things that already exist.

2. Straw and sticks will never be bricks.

Know that you're forced to build with weak materials (exploitable code) and design your structures accordingly. Any functionality may suddenly become a death trap or fire hazard, no matter what it was designed to be; so make sure such things can be amputated or walled off at a moment's notice.

3. Airliners should not attempt aerobatics.

Know that you are human, and are building with straw and sticks. Don't build death-defying skyscrapers that pose deliberate and unnecessary risks to homeowners. In particular, don't build in facilities that allow arbitrary passing wolves to overpower residents in their houses, even if that is appropriate design when you are building hotels owned by wolves.

Risky tricks like DRM, product activation, linking real-time WGA to DoS payloads etc. have no business in teetering edifices built from twigs.

4. Expose wildcard teams to new ideas.

Microsoft gets better at what they do well, while remaining poor at what they do poorly, or on issues to which they remain oblivious. Why is this?

Partly this is from over-reliance on rich but historical data, as per my first conclusion in this list. But it is also because their ability to develop is structured by present resource commitments. For something they already do, there will be a product team; any idea on how to do that stuff better may reach this team, who will understand what it's about and can swiftly incorporate such feedback.

But if they've never seen the need for something, they will have never formed a team to develop it. Any feedback on such matters will fall on dry ground; there is literally non-one there to process such material.

5. Handle unstructured feedback.

Microsoft regularly solicits feedback via surveys etc. but once again, these measure what is measurable, rather than what's important - so the objective of "getting new ideas" remains un-met.

Yes, it's easier to capture data gathered as responses to radio buttons, checkboxes, ordered pick lists and yes/no questions - but that limits input to what the designers of the survey had thought of already.

At the very least, every survey should end with a generic question such as: "On a scale from X to Y, how well do you think this survey covers what you feel should be surveyed?" followed by a large empty "comments" text box.

A high dissatisfaction score should warn you that you are digging in the wrong place; the response might be to form a wild-card team and pass the dissatisfied returns to them for assessment of any free-form comments that may give a sense of where you should be digging instead.

6. The cheapest lunch is where you haven't looked yet.

The saying "whenever I lose something, it's always in the last place I'd think to look for it!" is a truism, because once you find it, you stop looking. In fact, "lost things" are just the least successful tail of your usual access methods.

You stop looking when you find an answer, but that doesn't mean you have the best answer - there may be better ones if you'd look a bit further.

With a mature product that still has problems, the biggest gains are most likely to be found where no-one's started looking yet - rather than improving existing strengths past the point of diminishing returns.

7. In the land of the blind, name tags aren't useful.

The Internet is an unbounded mesh of strangers, so identity-based solutions don't apply. Once you initiate networking, as opposed to generic Internet access (e.g. after you log into a secure site), such solutions may become useful... but even then, only if the user has a template of expectations to match whatever identity has been proven. Even then, the process is only as robust as the twigs out of which it is built, and the Internet is a very windy place.

For this reason, I'd rate risk management as more important for malware and safety, both out on the web and within the system. This is the barely-touched area that is most likely to provide your cheapest lunch.

8. Who are the wolves?

All pigs become wolves in the dark.

We are the wolves, and so are you. There's no such thing as a special set of saintly piggies (e.g. "media content providers", "software vendors", etc.) who can be trusted with raw pork.

Design vs. Code Errors

2007-08-28T00:23:00.001-07:00

Technorati tags: Bug, Safety

When Microsoft finds a code error, it generally fixes this fairly promptly.

In contrast, design errors generally remain unfixed for several generations of products; sometimes years, sometimes decades. Typically even when addressed, the original design will be defended as "not an error" or "works as designed".

Old ideas that don't fit

As an example of bad design that has persisted from the original Windows 95 through to Vista, consider the in appropriateness of Format on the top layer of the Drive context menu.

The logic is old, and still true; hard drives are disks, and formatting is something you do to disks, therefore etc.

But around this unchanged truism, other things have changed.

We now have more things we can do to disks, many of which should be done more often than they are; backup, check for errors, defrag. Because these are "new" (as at Windows 95), they are tucked several clicks deeper in the UI, e.g. Properties, Tools.

Also, the word "Format" has some to mean different things to users. In 1985, users would routinely buy blank diskettes that had to be formatted before use, and so the immediate meaning of the word "format" was "to make a disk empty by destroying all existing contents". In 2007, users store things on USB sticks or optical disks, none of which have to be formatted (unless you use packet writing on RW disks) and the immediate meaning of the word "format" is "to make pretty", as in "auto-format this Word document" and "richly-formatted text".

The goal of software is to abstract the system towards the user's understanding of what they want to do. In keeping with this, "hard drives" have taken on a different conceptual meaning, away from the system reality of disks, towards an abstracted notion of "where things go". In particular, modern Windows tends to gloss over paths, directories etc. with conceptual locations such as "the desktop", "documents" etc. and the use of Search to find things vs. formal file system navigation across disks and directories.

New things that break old truths

When a risk doesn't arise due to hard scopes, one doesn't have to consider it. For example, if you build a house with a mountain as your back wall, you don't have to think about burglar-proofing the back wall. For example, if your LAN is cable-only in a physically-secured building, you have less worries about intrusion than if you'd added WiFi to the mix.

When a risk doesn't arise because a previous team anticipated and definitively fixed it, future teams may be oblivious to it as a risk. As Windows is decades old, and few programmers stay at the rock face for decades without being promoted to management or leaving, there's a real risk that today's teams will act as "new brooms", sweeping the platform into old risks.

In many of these cases, the risks were immediately obvious to me:

\Autorun.inf processing of hard drive volumes
Auto-running macros in "documents"
Active content in web pages

In some cases, I missed the risk until the first exploit:

Unfamiliar .ext and scripting languages

But it generally takes none to one exploit example for me to get the message, and take steps to wall out that risk. Alas, Microsoft keeps digging for generations:

MS Office 95, 97, 2000 after Concept, CAP, Thus etc.
OE in Internet Zone in WinME after BubbleBoy, Kak, Valentine

Auto-binding File and Print Sharing to DUN in Win9x, the way WiFi has been rolled out, dropping "network client" NT into consumerland as XP, hidden admin shares, exposing LSASS and RPC without firewall protection, encouraging path-agnostic file selection via Search... all of these are examples of changes that increase exposure to old risks, and/or new brooms that undermine definitive solutions as delivered by previous teams.

For example, the folks who designed DOS were careful to ensure that the type of file would always be immediately visible via the file name extension, limiting code types to .COM, .EXE and .BAT, and they were careful to ensure every file had a unique filespec, so that you'd not "open" the wrong one.

These measures basically solved most malware file-spoofing problems, but subsequent teams hide file name extensions, apply poor file type discipline, dumb "run" vs. "view"/"edit" down to the meaningless "open", act on hidden file type info without checking this matches what the user saw, and encourage searching for files that may pull up the wrong filespec.

Avoiding bad design

How would I prevent bad designs reaching the market, and thus creating an installed vendor/user base that create problems when the design is changed?

Keep core safety axioms in mind
Maintain old/new team continuity
Reassess logic of existing practices
Don't force pro-IT mindset on consumers
Assume bad intent for any external material
Make no assumptions of vendor trustworthiness

The classic safe hex rules...

Nothing runs on this system unless I choose to run it
I will assess and decide on all content before running it

...seem old and restrictive, but breaking these underlies most malware exploits.

The Word Is Not The World

2007-08-27T21:49:00.001-07:00

We can't use language to describe "the all".

Stated as baldly, this looks rather Zen, doesn't it?

The point being that language goes about defining particulars, i.e. "is this, is not that", and thus chips its way away from "the all".

In number theory terms, it's the difference between infinity and very large; of a limit, and test values that tend towards that limit.

The Waking Hour

Norton Life Sentence

2007-08-17T02:26:00.001-07:00

Technorati tags: Antivirus, Safety

This post is about Packard Bell, Norton Antivirus, Norton Internet Security and OEM bundling. For many readers, those four are all "yuk" items already...

Formal maintenance

Every "WTF" (i.e. ill-defined complaints, or just in an unknown state) PC that comes in, gets the formal treatment; 24 hours of MemTest86 with substituted boot CDR to detect spontaneous reboots, Bart boot HD Tune, and Bart booted formal malware scans.

This laptop cannot perform the RAM test because it keeps switching itself off, presumably because it is "idle" (no keyboard, HD, CD, LAN, mouse etc. interrupts). CMOS Setup shows no facility to manage such behavior, which is disabled in Windows already. Strike 1, Packard Bell.

The hard drive and file system are fine, and absolutely no malware at all were found on multiple formal av and anti-"spyware" scans, nor in four anti-"spyware" scans done in Safe Cmd. Spybot did note that the three Windows Security Center alerts were overridden, and this was later confirmed to be a Norton effect.

Specs and software

This is a fairly high-spec laptop; Mobile Celeron at 1.5GHz, 1G RAM (!), XP Pro SP2, but puny 45G hard drive with 4G stolen for the OEM's "special backup" material. The date stamp on the Windows base directory is 17 March 2006, which matches that of the SVI and "Program Files" directories too.

It has Norton Internet Security 7.0.6.17 OEM(90) and Norton Antivirus 2004 10.0.1.13 OEM(90). That's from the Help in these products; the same Help describes using Add/Remove to uninstall them.

Attempted uninstall

Both programs are definitely present and running; in fact, one gets nags every few minutes about antivirus being out of date, and firewall being disabled. A check confirms both to be true; neither Norton nor XP firewall is enabled, and Norton's subscription has expired.

However, Add/Remove shows no Norton entries other than Live Update. In fact, the expected slew of OEM bundleware are not there. A lethally-ancient Sun Java JRE 1.4.xx was found and uninstalled.

Start Menu shows an "Internet and security" flyout with icons for Norton Antivirus and Internet Security. No icons to uninstall these products from there.

What I did find in a "Packard Bell Support" Start Menu flyout, was a Smart Restore center, from which bundleware could be highlighted and installed or uninstalled. There was an alert to disable Norton's protection before doing this (more on that later), but either way, clicking Uninstall did nothing (no visible UI effect) and clicking OK after that, appeared to install Norton 2004 again.

To be continued...

Duplicate User Accounts

2007-08-15T01:19:00.001-07:00

Technorati tags: User Account

On Sat, 11 Aug 2007 20:58:01 -0700, SteveS

>My laptop is from Fujitsu and it came with OmniPass software (the
>fingerprint scanner software to log in). I saw other postings elsewhere
>about it duplicating users on the login screen. I uninstalled the software,
>rebooted - problem fixed (no more duplicate users). I reinstalled the
>software, rebooted, the duplicate users did not show up. I think it stems
>from the upgrade I did from Home Premium to Ultimate and had that software
>installed.

Yes; any "repair install" of XP will prompt you to create new user accounts even though you already have user accounts, and registry settings that clearly indicate these accounts are in use.

If you then enter the same name(s) as existing accounts, then new accounts are created with the same name.

Vista may avoid this conundrum, but fall into others.

Behind the scenes, the real names are not the same, because the real names are something quite different to what Windows shows you. Messy, but key to the ability of preserving continuity while allowing you to change the account name after it's created.

Specifically, you encounter not one, nor two, but three name sets:
- the "real" unique identifier, of the form S-n-n-nn-nnn...
- the name of the account base folder in Users or D&S
- the name as seen at logon on when managing users

In the case of account duplication in XP, you will have:
- unique and unrelated S-n-n-nn-nnnn... identifiers
- old Name and new Name.PCName account folders
- the same name at logon and account management

The risks of deleting the wrong material should be obvious.

Public Conversations

Malware: Avoid, Clean, or Rebuild?

2007-08-15T01:03:00.001-07:00

Technorati tags: Malware

On Sun, 12 Aug 2007 09:58:03 -0700, MrSlartybartfast

>Yes, creating an image of a hard drive which has malware would include the
>malware in the image. When copying this image back to the hard drive, the
>malware would also be copied back resulting in net gain of zero.

This is why "just backup!" (as glibly stated) is as useless as "just don't get viruses!" or "if you get infected, clean the virus!" etc.

All of these approaches work, but have complexity within them that make for YMMV results. The complexity is similar across all three contexts; how one scopes out the bad guys. The mechanics of meeting that inescapable challenge vary between the three "solutions".

>When I reinstall Windows, I reinstall off the original DVD which has
>no malware, unless you call Windows itself malware :)

This is using time as the great X-axis, i.e. the OS code base is as old as possible, therefore excludes the malware. And so, the PC is known to be clean.

But it also lacks every code patch needed to keep it that way, in the face of direct exploits a la Lovesan or Sasser etc. and to patch those, you'd have to expose this unpatched PC to the Internet.

It's also bereft of any applications and data. Presumably once can do the same with applications and drivers as with the OS; install known-good baseline code from CDs and then patch these online, or re-download apps and drivers from the 'net.

There's also no data, and another cruch comes here, because you probably don't want a data set that's certain to be too old to be infected; you want your most recent backup, which is the one most likely to be malware-tainted. How to scope data from malware?

Even though MS pushes "just" wipe and rebuild as the malware panacea, they undermine these poiunts of failure:
- they generally don't ship replacement code on CDs or DVDs
- they don't attempt to separate data, code and incoming material

The first has improved, what with XP SP2 being released as a CD, and with XP SP2 defaulting to firewall on.

There's little or no progess on the second, though; still no clearly visible distinction between data and code, still no type discipline so malware can sprawl across file types and spoof the user and OS into trusting these, incoming material is still hidden in mail stores and mixed with "documents" etc.

In Vista, just what is backed up and what is not is even more opaque, as there's little or no scoping by location at all.

>If the malware is on drive D:\ then it possibly could be reactivated on to
>drive C:\. You normally need to access the files on D:\ to reactivate the
>malware.

For values of "you" that includes the OS as a player. Even with a wipe-and-rebuild that ensures no registry pointers to code on D:, there can still be code autorun from D: via Desktop.ini, \Autorun.inf, or the exploitation of any internal surfaces.

Such surfaces may present themselves to the material:
- when you do nothing at all, e.g. indexers, thumbnailers etc.
- when you "list" files in "folders"
- when a file name is displayed

>No antivirus is perfect either, antivirus programs can often miss finding
>some malware. I tend to find antivirus programs clunky and annoying and
>prefer not to use them.

I use them, as I think most users do. If you "don't need" an av, then clearly you have solved the "don't get viruses" problem, and the contexts of "clean the virus" and "rebuild and restore data" don't arise. If they do arise, you were wong in thinking "don't get viruses" was solved, and maybe you should rethink "I don't need an av" (while I do agree that av will miss things).

Your nice freshly-built PC has no av, or an av installed from CD that has an update status far worse than whatever was in effect when you were infected. To update the av, you have to take this clean, unpatched, un-protected-by-av system online...

>On my D:\ I compress my files individually which makes it hard for malware
>to emerge.

That helps. It also helps in av can traverse this compression for the on-demand scans you'd want to do between rebuilding C: and installing and updating av, and doing anythiing on D: or restoring "data".

>It is a painful process and takes a few hours so I do not do this very often.

I should hope not; it's "last resort". If you have no confidence in the ability to detect or avoid malware, do you do this just when convenient, or whenever you "think you might be infected", or do you do it every X days so attackers have "only" X days in which they can harvest whatever they can grab off your PC?

>I do find this much easier than trying to live with an antivirus
>program installed. My choice is not for everyone

It might have been a best-fit in the DOS era, when "don't get viruses" was as easy as "boot C: before A: and don't run .EXE, .COM and .BAT files". By now, a single resident av poses little or no system impact, whereas the wipe-and-rebuild process is a PITA.

Frankly, doing a wipe-and-rebuild every now and then on a PC that's probably clean anyway, will increase the risks of infection.

Do the maths; you either get infected so often that the risks of falling back to unpatched code hardly makes things worse, in which case whatever you (blindly) do is equally useless, or your approach works so well that falling back to unpatched code is your single biggest risk of infection, and to improve things, you should stop doing that. If you have no ability to tell whether you are or have ever been infected, you can't distingusish between these states.

>as I said before I have no valuable information stored on
>my PC, I do not own a credit card and do not use internet
>banking. If I have malware then I can live with it.

Most of us want better results than that, and generally attain them.

Why are we reading this advice again?

>The AUMHA forum you linked to as a recommendation for Nanoscan and Totalscan
>does nothing for me, it is hardly a review. Panda Software is well known, so
>this is not one of the fake virus scans which is on the web. Out of
>curiosity I started to run it anyway, I did not continue since I do not yet
>fully understand the software and am not prepared to install the files on my
>PC. You may use this if you wish but it is not for me.

I agree with you there, especially if you suspect the PC is infected. How do you know the site you reached, is not a malware look-alike that resident malware has spoofed you to? Is it really a good idea to...
- disable resident av
- run Internet Explorer in admin mode so as to drop protection
- say "yes" to all ActiveX etc. prompts
- allow the site to drop and run code
- stay online while this code "scans" all your files
...as the advice at such sites generally suggests?

>The bots which harvest email addresses off the internet are just that, bots.
> They scour the entire internet, not just microsoft newsgroups. To be safe,
>never use your real name, never give your address, phone number or contact
>details, create temporary email accounts to use to sign up to forums and
>newsgroups,

Bots are unbounded, because:
- they can update themselves
- they facilitate unbounded interaction from external entities

Those external entities may be other bots or humans. In essence, an active bot dissolves confidence in the distinction between "this system" and "the Internet" (or more more accurately, "the infosphere", as local attacks via WiFi may also be facilitated).

Public Conversations

New User Account Duhfaults

2007-08-14T02:17:00.001-07:00

Technorati tags: User Account, XP, Safety, Risk Management

From...

http://www.spywarepoint.com/forums/t26963-p7-microsoft-zero-day-security-holes-being-exploited.html

On Thu, 28 Sep 2006 21:24:32 -0600, Dan wrote:
>cquirke (MVP Windows shell/user) wrote:

>> Defense in depth means planning for how you get your system back; you
>> don't just faint in shock and horror that you're owned, and destroy
>> the whole system as the only way to kill the invader.

>> It's absolutely pathetic to have to tell posters "well, maybe you have
>> 'difficult' (i.e., compitently-written) malware; there's nothing you
>> can do, 'just' wipe and re-install" because our toolkit is bare.

>The school computers (XP Pro. ones -- the school also has 98SE
>computers) where I work were all configured by someone who did
>not know what they were doing. They are have the remote assistance
>boxes checked and that is like saying to everyone "come on in to this
>machine and welcome to the party" This setting is just asking for
>trouble and yet the person or people who originally set up these
>machines configured them in this manner.

All your setup dudes did wrong was to install the OS while leaving MS duhfaults in place. By duhfault, XP will:
- full-share everything on all HDs to networks (Pro, non-null pwds)
- perform no "strength tests" on account passwords (see above)
- disallow Recovery Console from accessing HDs other than C:
- disallow Recovery Console from copying files off C:
- wave numerous services e.g. RPC, LSASS at the Internet
- do so with no firewall protection (fixed in SP2)
- allow software to disable firewall
- automatically restart on all system errors, even during boot
- automatically restart on RPC service failures
- hide files, file name extensions and full directory paths
- always apply the above lethal defaults in Safe Mode
- facilitate multiple integration points into Safe Mode
- allow dangerous file types (.EXE, etc.) to set their own icons
- allow hidden content to override visible file type cues
- dump incoming messenger attachments in your data set
- dump IE downloads in your data set
- autorun code on CDs, DVDs, USB storage and HD volumes
- allow Remote Desktop and Remote Assistance through firewall
- allow unsecured WiFi
- automatically join previously-accepted WiFi networks
- waste huge space on per-user basis for IE cache
- duplicate most of the above on a per-account basis
- provide no way to override defaults in new account prototype

Every time one "just" reinstalls Windows (especially, but not always only, if one formats and starts over), many or all of the above settings will fall back to default again. Couple that with a loss of patches, and you can see why folks who "just" format and re-install, end up repeating this process on a regular basis.

Also, every time a new user account is created, all per-account settings start off with MS defaults and you have to re-apply your settings all over again. If you limit the account rights, as we are urged to do, then often these settings lip back to MS defaults and remain there - so I avoid multiple and limited user accounts altogether, and prefer to impose my own safety settings.

>-- Risk Management is the clue that asks:
"Why do I keep open buckets of petrol next to all the
ashtrays in the lounge, when I don't even have a car?"
>----------------------- ------ ---- --- -- - - - -

Public Conversations

Free Users Need Control!

2007-08-14T02:10:00.001-07:00

Technorati tags: XP, malware, safety, maintenance OS

From...

http://www.spywarepoint.com/forums/t26963-p7-microsoft-zero-day-security-holes-being-exploited.html

On Tue, 26 Sep 2006 07:46:22 -0400, "karl levinson, mvp"

>All operating systems do that. They are designed to launch code at boot
>time by reading registry values, text files, etc. Because those registry
>values are protected from unauthorized access by permissions, someone would
>have to already own your system to modify those values, wouldn't they?

Sure, but the wrong entities come to own systems all the time. Defense in depth means planning for how you get your system back; you don't just faint in shock and horror that you're owned, and destroy the whole system as the only way to kill the invader.

It's tougher for pro-IT, because they've long been tempted into breaking the rule about never letting anything trump the user at the keyboard. By now, they need remote access and admin, as well as automation that can be slid past the user who is not supposed to have the power to block it, in terms of the business structure.

But the rest of us don't have to be crippled by pro-IT's addiction to central and remote administration, any more than a peacetime urban motorist needs an 88mm cannon in a roof-top turret. We need to be empowered to physically get into our systems, and identify and rip out every automated or remotely-intruded PoS that's got into the system.

It's absolutely pathetic to have to tell posters "well, maybe you have 'difficult' (i.e., compitently-written) malware; there's nothing you can do, 'just' wipe and re-install" because our toolkit is bare.

Public Conversations

On User Rights, Safe Mode etc.

2007-08-14T02:03:00.001-07:00

Technorati tags: Safety, Safe Mode, XP, malware, maintenance OS

Edited for spelling; from...

http://www.spywarepoint.com/forums/t26963-p8-microsoft-zero-day-security-holes-being-exploited.html

On Fri, 29 Sep 2006 23:17:02 -0400, "Karl Levinson, mvp"
>"cquirke (MVP Windows shell/user)" wrote in

>>>All operating systems do that. They are designed to launch code at boot
>>>time by reading registry values, text files, etc. Because those registry
>>>values are protected from unauthorized access by permissions, someone
>>>would have to already own your system to modify those values, wouldn't they?

The weakness here is that anything that runs during the user's session is deemed to have been run with the user's intent, and gets the same rights as the user. This is an inappropriate assumption when there are so many by-design opportunities for code to run automatically, whether the user intended to do so or not.

>> Sure, but the wrong entities come to own systems all the time.

>My point is that this one example here doesn't seem to be a vulnerability if
>it requires another vulnerability in order to use it.

Many vulnerabilities fall into that category, often because the extra requirement was originally seen as sufficient mitigation. Vulnerabilities don't have to facilitate primary entry to be significant; they may escalate access after entry, or allow the active malware state to persist across Windows sessions, etc.

>This isn't a case of combining two vulnerabilities to compromise a
>system; it's a case of one unnamed vulnerability being used to
>compromise a system, and then the attacker performs some other
>action, specifically changing registry values.

>If this is a vulnerability, then the ability of Administrators to create new
>user accounts, change passwords etc. would also be a vulnerability.

OK, now I'm with you, and I agree with you up to a point. I dunno where the earlier poster got the notion that Winlogin was there to act as his "ace in the hole" for controlling malware, as was implied.

>> Defense in depth means planning for how you get your system back; you
>> don't just faint in shock and horror that you're owned, and destroy
>> the whole system as the only way to kill the invader.

>That's a different issue than the one we were discussing. The statement
>was, winlogon using registry values to execute code at boot time is a
>vulnerability. I'm arguing that it is not.

I agree with you that it is not - the problem is the difficulty that the user faces when trying to regain control over malware that is using Winlogin and similar integration points.

The safety defect is that:
- these integration points are also effective in Safe Mode
- there is no maintenance OS from which they can be managed

We're told we don't need a HD-independent mOS because we have Safe Mode, ignoring the possibility that Safe Mode's core code may itself be infected. Playing along with that assertion, we'd expect Safe Mode to disable any 3rd-party integration, and would provide a UI through which these integration points can be managed.

But this is not the case - the safety defect is that once software is permitted to run on the system, the user lacks the tools to regain control from that software. Couple that with the Windows propensity to auto-run material either be design or via defects, and you have what is one of the most common PC management crises around.

>Besides, it's a relatively accepted truism that once an attacker has root,
>system or administrator privileges on any OS, it is fairly futile to try to
>restrict what actions s/he can perform. Anything a good administrator can
>do, a bad administrator can undo.

That's a safety flaw right there.

You're prolly thinking from the pro-IT perspective, where users are literally wage-slaves - the PC is owned by someone else, the time the user spends on the PC is owned by someone else, and that someone else expects to override user control over the system.

So we have the notion of "administrators" vs. "users". Then you'd need a single administrator to be able to manage multiple PCs without having to actually waddle over to all those keyboards - so you design in backdoors to facilitate administration via the network.

Which is fine - in the un-free world of mass business computing.

But the home user owns their PCs, and there is no-one else who should have the right to usurp that control. (Even) creditors and police do not have the right to break in, search, or seize within the user's home.

So what happens when an OS designed for wage-slavery is dropped into free homes as-is? Who is the notional "administrator"? Why is the Internet treated as if it were a closed and professionally-secured network? There's no "good administrators" and "bad administrators" here; just the person at the keyboard who should have full control over the system, and other nebulous entities on the Internet who should have zero control over the system.

Whatever some automated process or network visitation has done to a system, the home user at the keyboard should be able to undo.

Windows XP Home is simply not designed for free users to assert their rights of ownership, and that's a problem deeper than bits and bytes.

Public Conversations

On Win9x, SR, mOS II, etc.

2007-08-14T01:55:00.001-07:00

Technorati tags: Safety, malware, maintenance OS, Bart

Lifted from ...

http://www.spywarepoint.com/forums/t26963-p9-microsoft-zero-day-security-holes-being-exploited.html

On Sun, 01 Oct 2006 20:45:23 -0600, "Dan W." <spamyou@user.nec> wrote:
>karl levinson, mvp wrote:
>> "Dan W." <spamyou@user.nec> wrote in message

>> Fewer vulnerabilities are being reported for Windows 98 because Windows 98
>> is old and less commonly used, and vulns found for it get you less fame

More to the point is that vulnerable surfaces are less-often exposed to clickless attack - that's really what makes Win9x safer.

You can use an email app that displays only message text, without any inline content such as graphics etc. so that JPG and WMF exploit surfaces are less exposed. Couple that with an OS that doesn't wave RPC, LSASS etc. at the 'net and doesn't grope material underfoot (indexing) or when folders are viewed ("View As Web Page" and other metadata handlers) and you're getting somewhere.

For those who cannot subscribe to the "keep getting those patches, folks!" model, the above makes a lot of sense.

>> Didn't XP expand on and improve the system restore feature to a level not
>> currently in 98 or ME?

There's no SR in Win98, tho that was prolly when the first 3rd-party SR-like utilities started to appear. I remember two of these that seemed to inform WinME-era SR design.

No-one seemed that interested in adding these utilities, yet when the same functionality was built into WinME, it was touted as reason to switch to 'ME, and when this functionality fell over, users were often advised to "just" re-install to regain it. I doubt if we'd have advised users to "just" re-install the OS so that some 3rd-party add-on could work again.

XP's SR certainly is massively improved over WinME - and there's so little in common between them that it's rare one can offer SR management or tshooting advice that applies to both OSs equally.

I use SR in XP, and kill it at birth in WinME - that's the size of the difference, though a one-lunger (one big doomed C: installation may find the downsides of WinME's SR to less of an issue.

>>> about Microsoft and its early days to present time. The early Microsoft
>>> software engineers nicknamed it the Not There code since it did not have
>>> the type of maintenance operating system that Chris Quirke, MVP fondly
>>> talks about in regards to 98 Second Edition.

>> If the MOS being discussed for Win 98 is the system boot disk floppy, that
>> was a very basic MOS and it still works on Windows XP just as well as it
>> ever did on Windows 98. [Sure, you either have to format your disk as FAT,
>> or use a third party DOS NTFS driver.]

That was true, until we crossed the 137G limit (where DOS mode is no longer safe). It's a major reason why I still avoid NTFS... Bart works so well as a mOS for malware management that I seldom use DOS mode for that in XP systems, but data recovery and manual file system maintenance remain seriously limited for NTFS.

>> I think Chris really wants not that kind of MOS but a much bigger and
>> better one that has never existed.

Well, ever onward and all that ;-)

Bart is a bigger and better mOS, though it depends on how you build it (and yes, the effort of building it is larger than for DOS mode solutions). You can build a mOS from Bart that breaks various mOS safety rules (e.g. falls through to boot HD on unattended reset, automatically writes to HD, uses Explorer as shell and thus opens the risk of malware exploiting its surfaces, etc.).

I'm hoping MS WinPE 2.0, or the subset of this that is built into the Vista installation DVD, will match what Bart offers. Initial testing suggests it has the potential, though some mOS safety rules have been broken (e.g. fall-through to HD boot, requires visible Vista installation to work, etc.).

The RAM testing component is nice but breaks so many mOS safety rules so badly that I consider it unfit for use:
- spontaneous reset will reboot the HD
- HD is examined for Vista installation before you reach the test
- a large amount of UI code required to reach the test
- test drops the RAM tester on HD for next boot (!!)
- test logs results to the HD (!!)
- you have to boot full Vista off HD to see the results (!!!)

What this screams to me, is that MS still doesn't "get" what a mOS is, or how it should be designed. I can understand this, as MS WinPE was originally intended purely for setting up brand-new, presumed-good hardware with a fresh (destructive) OS installation.

By default, the RAM test does only one or a few passes; it takes under an hour or so - and thus is only going to detect pretty grossly-bad RAM. Grossly bad RAM is unlikely to run an entire GUI reliably, and can bit-lip any address to the wrong one, or any "read HD" call to a "write HD" call. The more code you run, the higher the risk of data corruption, and NO writes to HD should ever be done while the RAM is suspected to be bad (which is after all why we are testing it.

A mOS boot should never automatically chain to HD boot after a time out, because the reason you'd be using a mOS in the first place is because you daren't boot the HD. So when the mOS disk boots, the only safe thing to do is quickly reach a menu via a minimum of code, and stop there, with no-time-out fall-through.

It's tempting to fall-through to the RAM test as the only safe option, but that can undermine unattended RAM testing - if the system spontaneously resets during such testing, you need to know that, and it's not obvious if the reboot restarts the RAM test again.

Until RAM, physical HD and logical file system are known to be safe, and it's known that deleted material is not needed to be recovered, it is not safe to write to any HD. That means no page file, no swap, and no "drop and reboot" methods of restarting particular tests.

Until the HD's contents are known to be malware-free, it is unsafe to run any code off the HD. This goes beyond not booting the HD, or looking for drivers on the HD; it also means not automatically groping material there (e.g. when listing files in a folder) as doing so opens up internal surfaces of the mOS to exploitation risks.

Karl's right, tho... I'm already thinking beyond regaining what we lost when hardware (> 137G, USB, etc.) and NTFS broke the ability to use DOS mode as a mOS, to what a purpose-built mOS could offer.

For example, it could contain a generic file and redirected-registry scanning engine into which av vendor's scanning modules could be plugged. It could offer a single UI to manage these (i.e. "scan all files", "don't automatically clean" etc.) and could collate the results into a single log. It could improve efficiency by applying each engine in turn to material that is read once, rather than the norm of having each av scanner pull up the material to scan.

MS could be accused of foreclosing opportunities to av vendors (blocking kernel access, competing One Care and Defender products), but this sort of mOS design could open up new opportunities.

Normally, the av market is "dead man's shoes"; a system can have only one resident scanner, so the race is on to be that scanner (e.g. OEM bundling deals that reduce per-license revenue). Once users have an av, it becomes very difficult to get them to switch - they can't try out an alternate av without uninstalling what they have, and no-one wants to do that. It's only when feeware av "dies" at the end of a subscription period, that the user will consider a switch.

But a multi-av mOS allows av vendors to have their engines compared, at a fairly low development cost. They don't have to create any UI at all, because the mOS does that; all they have to do is provide a pure detection and cleaning engine, which is their core compitency anyway.

Chances are, some av vendors would prefer to avoid that challenge :-)

>> XP also comes with a number of restore features such as Recovery
>> Console and the Install CD Repair features.

They are good few-trick ponies, but they do not constitute a mOS. They can't run arbitrary apps, so they aren't an OS, and if they aren't an OS, then by definition that aren't a mOS either.

As it is, RC is crippled as a "recovery" environment, because it can't access anything other than C: and can't write to anywhere else. Even before you realise you'd have to copy files off one at a time (no wildcards, no subtree copy), this kills any data recovery prospects.

At best, RC and OS installation options can be considered "vendor support obligation" tools, i.e. they assist MS in getting MS's products working again. Your data is completely irrelevant.

It gets worse; MS accepts crippled OEM OS licensing as being "Genuine" (i.e. MS got paid) even if they provide NONE of that functionality.

The driver's not even in the car, let alone asleep at the wheel :-(

>> I never use those or find them very useful for security, but they're
>> way more functional and closer to an MOS than the Win98 recovery
>> floppy or anything Win98 ever had. 98 never had a registry
>> editor or a way to modify services like the XP Recovery Console.

They do different things.

RC and installation options can regain bootability and OS functionality, and if you have enabled Set commands before the crisis you are trying to manage, you can copy off files one at a time. They are limited to that, as no additional programs can be run.

In contrast, a Win98EBD is an OS, and can run other programs from diskette, RAM disk or CDR. Such programs include Regedit (non-interactive, i.e. import/export .REG only), Scandisk (interactive file system repair, which NTFS still lacks), Odi's LFN tools (copy off files in bulk, preserving LFNs), Disk Edit (manually repair or re-create file system structure) and run a number of av.

So while XP's tools are bound to getting XP running again, Win98EBD functionality encompasses data recovery, malware cleanup, and hardware diagnostics. It's a no-brainer as to which I'd want (both!)

>>> that at the bare bones level the source code of 9x is more secure

>> It depends on what you consider security.

That's the point I keep trying to make - what Dan refers to is what I'd call "safety", whereas what Karl's referring to is what I'd call "security". Security rests on safety, because the benefit of restricting access to the right users is undermined if what happens is not limited to what these users intended to happen.

>> Win98 was always crashing and unstable,

Er... no, not really. That hasn't been my mileage with any Win9x, compared to Win3.yuk - and as usual, YMMV based on what your hardware standards are, and how you set up the system. I do find XP more stable, as I'd expect, given NT's greater protection for hardware.

>> because there was no protection of memory space from bad apps or
>> bad attackers.

Mmmh... AFAIK, that sort of protection has been there since Win3.1 at least (specifically, the "386 Enhanced" mode of Win3.x). Even DOS used different memory segments for code and data, though it didn't use 386 design to police this separation.

IOW, the promise that "an app can crash, and all that happens is that app is terminated, the rest of the OS keeps running!" has been made for every version of Windows since Win3.x - it's just that the reality always falls short of the promise. It still does, though it gets a little closer every time.

If anything, there seems to be a back-track on the concept of data vs. code separation, and this may be a consequence of the Object-Orientated model. Before, you'd load some monolithic program into its code segment, which would then load data into a separate data segment. Now you have multiple objects, each of which can contain thier own variables (properties) and code (methods).

We're running after the horse by band-aiding CPU-based No-Execute trapping, so that when (not if) our current software design allows "data" to spew over into code space, we can catch it.

>> Microsoft's security problems have largely been because of backwards
>> compatibility with Windows 9x, DOS and Windows NT 4.0. They feel, and I
>> agree, that Microsoft security would be a lot better if they could abandon
>> that backwards compatibility with very old niche software, as they have been
>> doing gradually.

The real millstone was Win3.yuk (think heaps, co-operative multitasking). Ironically, DOS apps multitask better than Win16 ones, as each DOS app lives in its own VM and is pre-emptively multi-tasked.

64-bit is the opportunity to make new rules, as Vista is doing (e.g. no intrusions into kernel allowed). I'm hoping that this will be as beneficial as hardware virtualization was for NT.

Win9x apps don't cast as much of a shadow, as after all, Win9x's native application code was to be the same as NT's. What is a challenge is getting vendors to conform to reduced user rights, as up until XP, they could simply ignore this.

There's also the burden of legacy integration points, from Autoexec.bat through Win.ini through the various fads and fashions of Win9x and NT and beyond. There's something seriously wrong if MS is unable to enumerate every single integration point, and provide a super-MSConfig to manage them all from a single UI.

>Classic Edition could be completely compatible with the older software
>such as Windows 3.1 programs and DOS programs. Heck, Microsoft
>could do this in a heartbeat without too much trouble.

Think about that. Who sits in exactly the same job for 12 years?

All the coders who actually made Win95, aren't front-line coders at MS anymore. They've either left, or they've climbed the ladder into other types of job, such as division managers, software architects etc. To the folks who are currently front-line coders, making Vista etc., Win9x is as alien as (say) Linux or OS/2.

To build a new Win9x, MS would have to re-train a number of new coders, which would take ages, and then they'd have to keep this skills pool alive as long as the new Win9x were in use. I don't see them wanting to do that, especially as they had such a battle to sunset Win9x and move everyone over to NT (XP) in the first place.

Also, think about what you want from Win9x - you may find that what you really want is a set of attributes that are not inherently unique to Win9x at all, and which may be present in (say) embedded XP.

If you really do need the ability to run DOS and Win3.yuk apps, then you'd be better served by an emulator for these OSs.

This not only protects the rest of the system to the oddball activities of these platforms, but can also virtualize incompatible hardware and mimic the expected slower clock speeds more smoothly than direct execution could offer. This is important, as unexpected speed and disparity between instruction times is as much a reason for old software to fail on new systems as changes within Windows itself.

>I will do what it takes to see this come to reality.

Stick around on this, even if there's no further Win9x as such. As we can see from MS's first mOS since Win98 and WinME EBDs, there's more to doing this than the ability to write working code - there has to be an understanding of what the code should do in the "real world".

Public Conversations

CDRW/DVDRW Primer

2007-08-13T12:18:00.001-07:00

It can be a bit confusing figuring out R vs. RW and formal authoring vs. packet writing, but I'll try. This skips a lot of detail, and attempts to zoom on what you'd need to know if starting on writing CDs or DVDs in 2007...

Here's the executive summary:

	R	RW

Authored	Fine	Fine
Packet-written	Can't	Sucks

R vs. RW disks

R(ecordable) disks are like writing in ink - once you've written, you cannot erase, edit or overwrite.

R(e)W(ritable) disks are like writing in pencil - you can rub out what you want to change, but what you write in there, has to fit between whatever else you have not rubbed out.

Authoring vs. packet writing

The "authoring" process is like setting up a printing press; you first lay out the CD or DVD exactly as you want it, then you splat that onto the disk. You can fill the whole disk at once, like printing a book (single session), or you can fill the first part and leave the rest blank to add more stuff later, like a printed book that has blank pages where new stuff can be added (start a multisession).

The "packet writing" process is what lets you pretend an RW disk is like a "big diskette". Material is written to disk in packets, and individual packets can be rubbed out and replaced with new packets, which pretty much mirrors the way magnetic disks are used. This method is obviously not applicable to R disks.

RW disks can also be authored, but the rules stay the same; you either add extra sessions to a multi-session disk, or you erase the whole disk and author it all over again.

Overwriting

When you overwrite a file in a packet-writing system, you do so by freeing up the packets containing the old file and write the new file into the same and/or other packets. The free space left over is increased by the size of the old file and reduced by the size of the new, rounded up to a whole number of packets.

When you "overwrite" a file in a multisession (authored) disk, it is like crossing out the old material and writing new material underneath, as one is obliged to do when writing in ink. The free space drops faster, because the space of the old file cannot be reclaimed and re-used, and because each session has some file system overhead, no matter how small the content.

Standards and tools

There are a number of different standard disk formats, all of which must be formally authored; audio CDs, movie DVDs, CD-ROMs and DVD-ROMs of various flavors. In contrast, packet-written disk formats may be proprietary, and supported only by the software that created them.

Nero and Easy CD Creator are examples of formal authoring tools, and several media players can also author various media and data formats.

InCD and DirectCD are examples of packet-writing tools, which generally maintain a low profile in the SysTray, popping up only to format newly-discovered blank RW disks. The rest of the time, they work thier magic behind the scenes, so that Windows Explorer can appear to be able to use RW disks as "big diskettes".

Windows has built-in writer support, but the way it works can embody the worst of both authoring and packet-writing models. I generally disable this support and use Nero instead.

Flakiness

RW disks and flash drives share a bad characteristic; limited write life. In order to reduce write traffic to RW disks, packet writing software will hold back and accumulate writes, so these can be written back in one go just before the disk is ejected.

What this means is that packet written disks often get barfed by bad exits, lockups, crashes, and forced disk ejects. Typically the disk will have no files on it, and no free space. When this happens, you can either erase the disk and author it, or format the disk for another go at packet writing. Erasing is faster, while formatting applies only to packet writing (it defines the packets).

I have found that packet writing software has been a common cause of system instability (that often ironically corrupts packet-written disks). The unreliability, slow formatting, and poor portability across arbitrary systems have all led me to abandon packet writing in favor of formally authoring RW disks.

Back to Basics

Evolution vs. Intelligent Design

2007-08-09T01:23:00.001-07:00

Technorati tags: Information theory, memetics, evolution

Evolution vs. Intelligent Design = non-issue.

Evolution does not define why things happen.

It is a mechanism whereby some things that happen, may come to persist (and others, not).

Graded belief

Human thinking appears to have at least two weaknesses; an automatic assumption of dualities (e.g. "Microsoft and Google are both large; Microsoft is bad, therefore Google must be good"), and an unwillingness to accept unknowns.

You can re-state the second as a tighter version of the first, i.e. the singleton assumption, rather than duality.

We don't even have words (in English, at least) to differentiate between degrees of belief, i.e. weak ("all things equal, I think it is more likely that A of A, B, C is the truth") and strong ("iron is a metal") belief.

And I fairly strongly believe we strongly believe too often, when a weaker degree of certainty would be not only more appropriate, but is a needed component in our quest for Word Peace TM.

For example, religious folks have a fairly high certainty of what will happen after they die - but can we all accept that as they have not yet died, that something slightly less than "you're wrong, so I'll kill you" certainty should apply?

What is evolution?

My understanding of evolution, or Darwinian systems, revolves around the following components:

limited life span
selection pressure
imperfect reproduction

That's what I consider to be a classic evolutionary environment, but you may get variations; e.g. if entities change during the course of their lifetime, do not reproduce, but can die, then you could consider this as a Darwinian system that is ultimately set to run down like a wind-up clock as the number of survivors declines towards zero.

In fact, implicit in that classic model is the notion of reproduction based on a self-definition that does not change during the course of an entity's lifetime (in fact, it defines that entity) but can change when spawning next generation entities.

Evolution is blind

Evolution per se, is devoid of intent. I don't know whether Darwin stressed this in his original writings, but I weakly believe that he did; yet I often see descriptions of creatures "evolving to survive".

As I understand it, game theory is a reformulation of evolution that centers on the notion of survival intent.

Evolution is something that happens to things, and doesn't "care" whether those things survive or not. The "selfish gene" concept is an attempt to frame this inevitable sense of "intent" within Darwinian mechanics; there is no more need to ascribe a survival intent to genes, as there is to the phenotypes they define.

However, evolution doesn't have to be the only player on the stage, and this is what I meant about "evolution vs. intelligent design is a non-issue".

I don't think there's much uncertainty that evolution is at work in the world. That doesn't weigh for or against other (intelligent design) players in the world, and that's why I consider the question a non-issue.

Intelligent players can apply intent from outside the system (e.g. where an external entity defines entities within a Darwinian environment, or the environment itself, or its selection pressures) or from within the system (e.g. where entities apply intent to designing their own progeny).

Example systems

I consider the following to be Darwinian systems:

the biosphere
the infosphere
human culture, i.e. memetics

One could theorize that evolution is an inevitable consequence of complexity, when subjected to entropy. Just as a moving car is not fast enough to exhibit significant relativistic effects (so that Newton's laws appear to explain everything), so trivial systems may be insufficiently complex to demonstrate Darwinian behavior.

This is why I'm interested in computers and the infosphere; because they are becoming complex enough to defy determinism.

Normally, we seek to understand the "real world" by peering down from the top, with insufficient clarity to see the bottom.

With the infosphere, we have an environment that we understand (and created) from the bottom up; what we cannot "see" is the top level that will arise as complexity evolves.

This creates an opportunity to model the one system within the other. What was an inscrutable "mind / brain" question, becomes "the mind is the software, the brain is the hardware", perhaps over-extended to "the self is the runtime, the mind is the software, the brain is the firmware, the body is the hardware".

We can also look at computer viruses as a model for biosphere viruses. A major "aha!" moment for me was when I searched the Internet for information on the CAP virus, and found a lot of articles that almost made sense, but not quite - until I realized these described biological viruses, and were found because of the common bio-virus term "CAPsule".

Code

Common to my understanding of what constitutes a classic Darwinian system, is the notion of information that defines the entity.

In the biosphere, this is usually DNA or RNA, a language of 4 unique items grouped into threes to map to the active proteins they define.

In the infosphere, this is binary code of various languages, based on bits that are typically grouped into eights as bytes.

In the meme space, languages are carried via symbol sets that are in turn split into unique characters, which are then clumped into words or sentences. Some languages contain less information within the character set (e.g. Western alphabets), others more (e.g. the Chinese alphabet, ancient Egyptian hieroglyphics, modern icons and branding marks).

When we create computer code, we are laboriously translating the memetic language of ideas into code that will spawn infosphere entities. This is not unlike the way a set of chromosomes becomes a chicken, other than that we view the infosphere and meme space as separate Darwinian systems.

The alluring challenge is to translate infosphere code into biosphere code, i.e. to "print DNA", as it were. One hopes the quality of intent will be sound, by the time this milestone is reached, as in effect, we would be positioned to become our own intelligent creators.

Intelligent design

We know that entities in the infosphere are created by intent from outside the system; as at 2007, we do not believe that new entities arise spontaneously within the system.

We don't know (but may have beliefs about) whether there is intent applied to the biosphere, or whether the biosphere was originally created or shaped by acts of intent.

Conspiracy theorists may point to hidden uber-intenders within the meme space, the creation of which is inherently guided by self-intent.

That which was

Just as folks mistakenly ascribe intent to the mechanics of evolution, so there is a fallacy that all that exists, is all that existed.

But evolution can tell you nothing about what entities one existed, as spawned by mutation or entropic shaping of code.

There is a very dangerous assumption that because you cannot see a surviving entity in the current set, that such entities cannot arise.

Think of a bio-virus with a fast-death payload a la rabies, plus rapid spread a la the common cold. The assumption of survival intent leads folks to say stupid things like "but that would kill the host, so the virus wouldn't want that". Sure, there'd be no survivors in today's entity set, but on the other hand, we know we have some historical bulk extinctions to explain.

We're beginning to see the same complacency on the risks of nuclear war. We think of humanity as a single chain of upwards development, and therefore are optimistic that "common sense will prevail". Even nay-sayers that point to our unprecedented ability to destroy ourselves, miss the point in that word "unprecedented". As Graham Hancock postulated in his Fingerprints of the Gods, we may have been this way before.

This blindness applies to malware within the infosphere as well, and the saying "there's none so blind as those who can see" applies. If we want authoritative voices on malware, we generally turn to professionals who have been staring at all malware for years, such as the antivirus industry. These folks may be blinded by what they've seen of all that has been, that they fail to consider all that could be.

The Waking Hour

Low Heap Space in XP and Vista

2007-08-07T05:31:00.001-07:00

Technorati tags: Bug, XP, Vista

Have you ever been motoring along in XP or Vista, opening up new tabs in IE7, running apps in the background, etc. and noticed new tabs don't show the pages, or that when you right-click a link, you don't get a context menu? Or have you ever opened a couple of dozen photos in Irfan View and found the last few come up with no menu, and don't respond to hotkeys?

Did this bring back memories of "low resource heaps" in Win9x, or the need to restart DOS/Win3.yuk PCs several times a day to keep MS Office apps running properly?

You aren't going crazy, and yes, it's the same problem. It's like deja vu all over again.

Background

Windows 3.x used two or three "heaps", i.e. areas of RAM set aside for certain items that are spawned by running programs. There was a GDI heap for graphic elements, a user heap for UI and other elements, and a "system" heap that may have been a logical view of the first two (it's been a while, and I hoped I never have to remember the details again).

When folks started multitasking in earnest, these heaps would fill up and cause crashes, half-drawn UI elements, or spurious "out of memory" errors. Adding RAM was as useful as adding a trailer to a removal van with a loading crew so dumb they insist on putting all metal objects in the glove compartment and then tell you "we're full" after moving your hi-fi.

Windows 95 design intended to move these heaps to 32-bit replacements, but it was found that doing so would break several applications (including, it was rumored, Excel 4.0) which wrote directly to heap objects in memory rather than using the proper API calls.

So Windows 95 went into a long public beta where these things were thrashed out. Win9x left some items in "legacy" 16-bit heaps, and proactively cleared heap allocations when closing down VMs. 32-bit and DOS programs were run in their own VM, so this curbed "heap leakage" for these, but as 16-bit Windows all ran in a single shared VM, any resident 16-bit Windows program (e.g. Bitware 3.x) could hold this VM open forever.

Pundits would smugly point out that NT was a "true" 32-bit OS which did not make such compromises. The "huge" 32-bit address range should mean 32-bit heaps would never run out again.

The broken rule is...

"Do not use finite global storage for the per-instance data of an unbounded number of instances".

There are several instances of this rule being broken, such as the way Windows Explorer "remembers" settings for different folders.

The converse rule...

"Do not scale a per-instance resource based on evolving global store capacity"

...is broken by IE''s grabbing of X% of total HD volume space for its web cache, as well as System Restore capacity allocation. The first example was only fixed in IE7; the second, continues.

NT broke the first of these two scalability rules. The NT developers did not fix the heap problem via dynamically-sized heaps that could expand up to the limits of 32-bit addressability. Instead, they set various arbitrary limits at various times, in fact reducing these between versions 3.1 and 3.5 of the original NT.

The fix

Apparently this has been a known issue at Microsoft, though most of us (myself included) didn't see the old "resource heap blues" for the first few years on XP. I found some good background coverage in blogs and elsewhere, but the fix (adjusting certain registry settings) comes with some caveats:

Please do not modify these values on a whim. Changing the second or third value too high can put you in a no-boot situation due to the kernel not being able to allocate memory properly to even get Session 0 set up.

I haven't applied the fix yet, but probably will, on my XP SP2 system with IE7 etc. If it holds up here, and I have complaints from Vista users, then I may apply the fix to my new Vista PC builds as well.

Coverage

As you can guess, some blogs have been quite hostile to Microsoft, and this one is no exception (to me, it seems as if these folks missed the "Scalability 101" lecture).

However, I can see some reasons why they may have chosen not to create a dynamically-resizing heap system.

One reason is to prevent DoS memory usage through deliberate leaks; another might be concern over exploitable race conditions that might arise if one process is creating heap objects while another deliberately releases heap space to provoke a downwards resize.

There may be unacceptable overhead in managing more scalable data structures, such as linked lists, if heap objects are accessed as often as I suspect they might be.

Whatever such reasons might be, I'd suggest Microsoft get their story up really soon, before the trickle turns to a flood and users twig onto why they can't do "too many things at the same time" on their PCs - especially given the usual Vista mantra of "it's slow and needs more resources, but that's so it can scale up to future needs" that I've been known to wave around :-)

Seek, And Ye Shall Find... What?

2007-08-02T04:15:00.001-07:00

Technorati tags: safety

Re: Can a saved search be indistinguishable from real folders?

On Sat, 28 Jul 2007 05:34:00 -0700, Baffin

>To my mind, it would be elegant and correct for the operating system to
>present a 'saved search' folder to all appolications exactly as a real folder
>is presented. Thus no changes should be required to any applications -- they
>should all just work as usual -- instant access to saved-search collections
>-- great!

Think through the safety and security implications of that.

>But as mentioned originally, I'm having problems getting many applications
>to work with saved-search folders -- am I doing something wrong? Or, could
>Microsoft have implemented 'virtual folders' (saved searches)
>non-transparently? If so, why? What's the advantage worth all the
>disruption that would cause to applications?

I think there was a change in design intention on this.

Originally, when Vista was to embed SQL within the WinFS file system, these "virtual folders" were to function more transparently as folders, as you expect.

When WinFS was dropped, functionalities of "virtual folders" were scaled back - I'd thought they had been dropped alltogether.

Just as web mania drove MS to embed IE4 in Win98, with "View As Web Page" on your local file system, so search mania has driven MS to embed search into Vista.

Just as there were safety downsides to blurring the edge between Internet and local PC (as well as HTML-everywhere also dropping scriptability everywhere), so may there be safety downsides to searching for rather than specifying the files etc. you "open".

>Why can't it be 'invisible' to all existing applications whether or not a
>folder is real or 'virtual' (ie., a saved search)?

The original intention of file names was to ensure that every file was uniquely named. When this hit scalability issues, the concept of directories and paths was added.

The need to uniquely identify files is as strong as ever, in an age of pervasive malware, phishing, etc. but is also necessary to avoid "version soup" problems, reversion to pre-patch code, and confusion between old and new versions of data files that may be scattered across "live", backup, and off-PC storage locations.

When you throw away that specificity and just "search" for things, you need to be very sure about what you are looking at. Not easy, through a shell that hides file name extensions, allows dangerous file types to define their own icons, etc.

So yes; I *definately* want it to be very obvious as to whether I am looking at a directory, or some virtual collection of found items.

Public Conversations

When ChkDsk Doesn't

2007-08-02T04:00:00.001-07:00

Technorati tags: ChdDsk, file system, data recovery

Subject: Re: Unable to run CHKDSK with "Fix" option

On Wed, 25 Jul 2007 17:46:03 -0700, Paulie

I think it is beyond time that we had a proper interactive file system maintenance tool for NTFS. ChkDsk is a relic from the MS-DOS 5 days; I wish NT would at least catch up to, say, MS-DOS 6 Scandisk.

Now folks will flame me for saying that. "File systems are too complex for users to understand, just trust us to fix everything for you". Fine; let's read on and see how well that works...

>My new Notebook is unable to run a CHKDSK with the Fix option selected.
>I can run a normal CHKDSK within VISTA and it works without a problem.
>If I choose the Fix option it schedules a scan on the next boot. Upon
>rebooting,
>CHKDSK will begin but it will freeze after 8% of the scan. The
>Notebook does not respond and I have to power it down and restart.

Great, so now we combine a possibly corrupted file system in need of repair, with recurrent bad exits. What's wrong with this picture?

>Any idea what this may be?

Given that ChkDsk and AutoChk are closed boxes with little or no documentation of what they are doing (and little or no feedback to you while they are doing it), one can only guess.

My guesses would be one of:

1) Physically failing HD

When a sector can't be read, the HD will retry the operation a number of times before giving up. Whatever driver code that calls the operation will probably also retry a few times, before giving up, and so may the higher-level code that called that, etc.

The result can be an apparent "hang" lasting seconds to minutes while the system beats the dying disk to death.

That's before you factor in futile attepts to paper over the problem and pretend it isn't there, both by the HD itself, and by the NTFS code. Each will attempt to read the sick allocation unit's data and write it to a "good" replacement, then switch usage so that the dead sector is avoided in future. And so on, for next dead sector, etc.

2) Lengthy repair process

Scandisk and ChkDsk have no "big picture" awareness. If they were you, walking from A to B, they would take a step, calculate if they were at B, then take another, and repeat. If they were walking in the wrong direction, away from B, they'd just keep on walking forever.

So when something happens that invalidates huge chunks of the file system, these tools don't see the "big picture" and STOP and say "hey, something is invalidating the way this file system is viewed". No; they look at one atom of the file system, change it to fit the current view, and repeat for the next. If that means changing evey atom in the file system, that is what they will do. Result; garbage.

3) Bugginess

Whereas (2) is a bad design working as designed, sometimes the code doesn't work as designed and falls off the edge.

Needless to say, AutoChk and ChkDsk don't maintain any undoability. They also "know better than you", so they don't stop and ask you before "fixing" things, they just wade in and start slashing away.

>I have run full virus scans and updates on the drive and
>there are no issues. Other than this the Notebook runs fine. Its just a
>strange issue that i am unable to fix at this stage.

I would at least exclude (1) by checking the HD's surface using the appropriate tests in HD Tune (www.hdtune.com), after backing up my data. You should be able to get a "second opinion" on the file system, but you can't; ChkDsk and AutoChk are all you have.

Public Conversations

Malware Poll Results

2007-08-02T02:00:00.001-07:00

Technorati tags: Poll results

I played with Blogger's "poll" feature, asking whether malware was or was not reader's biggest PC headache. The results:

16% - Malware is my biggest PC headache
66% - Malware is not my biggest PC headache
0% - I'm OK, I use Linux / BSD / other
16% - I'm OK, I use a Mac
0% - I'm not on Windows, but malware is still a worry

That's on a tiny sample size of 12 respondents :-)

Public Conversations

2007-08-02T01:54:00.001-07:00

Technorati tags: Blogging

A lot of what I write is posts in transient newsgroups and forums, and although I resolved to write things "properly" in web pages and then blogs, I tend not to do this. I find it far easier to respond to another entity than write "cold" for a generalized audience.

As mentioned, I'm blending blogging with formal web site navigation using whatever tools come to hand. I'll use Blogger's labels as category selectors so that this blog can function as a set of unrelated blogs. These "virtual" blogs may include:

Public Conversations
The Waking Hour
Maintenance OS
Safety and Security

The first of these is Public Conversations, where I will simply paste what I see as interesting posts that are already public, edited only to allow natural line lengths and to break visible email addresses.

Over at my other blog, I've added a few new photo galleries, using "comments" to use these as pictorial articles rather than just a bunch of loose photos. So far, these include:

Both Blogger and Live Spaces have their strengths and weaknesses, so I'll probably continue adding new content to both.

Ship Now, Patch Later

2007-08-02T00:43:00.001-07:00

Technorati tags: Safety, security

Subject: Re: It would be nice if MS could settingle on a single subnet for updates

On Fri, 27 Jul 2007 15:13:52 +0100, "Mike Brannigan"
>"Leythos" <void@nowhere.lan> wrote in message
>> Mike.Brannigan@localhost says...

This thread is about the collision between...

No automatic code base changes allowed

...and...

Vendors need to push "code of the day"

Given the only reason we allow vendors to push "code of the day" is because their existing code fails too often for us to manage manually, one wonders if our trust in these vendors is well-placed.

A big part of this is knowing that only the vendor is pushing the code, and that's hard to be sure of. If malware were to hijack a vendor's update pipe, it could blow black code into the core of systems, right pas all those system's defenses.

With that in mind, I've switched from wishing MS would use open standards for patch transmission to being grateful for whatever they can do to harden the process. I'd still rather not have to leave myself open to injections of "code of the day", though.

>NO never ever ever in a production corporate environment do you allow ANY of
>your workstations and servers to directly access anyone for patches
>I have never allowed this or even seen it in real large or enterprise
>customers. (the only place it may crop up is in mom and pop
>10 PCs and a Server shops).

And there's the problem. MS concentrates on scaling up to enterprise needs, where the enterprise should consolodate patches in one location and then drive these into systems under their own in-house control.

So scaling up is well catered for.

But what about scaling down?

Do "mom and pop" folks not deserve safety? How about single-PC users which have everything they own tied up in that one vulnerable box? What's best-practice for them - "trust me, I'm a software vendor"?

How about scaling outwards?

When every single vendor wants to be able to push "updates" into your PC, even for things as trivial as prinyers and mouse drivers, how do you manage these? How do you manage 50 different ad-hoc update delivery systems, some from vendors who are not much beyond "Mom and Pop" status themselves? Do we let Zango etc. "update" themselves?

The bottom line: "Ship now, patch later" is an unworkable model.

>As you said your only problem is with Microsoft then the solution I have
>outlined above is the fix - only one server needs access through your
>draconian firewall policies. And you get a real secure enterprise patch
>management solution that significantly lowers the risk to your environment.

That's prolly the best solution, for those with the resources to manage it. It does create a lock-in advantage for MS, but at least it is one that is value-based (i.e. the positive value of a well-developed enterprise-ready management system).

However, I have to wonder how effective in-house patch evaluation really is, especially if it is to keep up with tight time-to-exploit cycles. It may be the closed-source equivalent of the open source boast that "our code is validated by a thousand reviewers"; looks good on paper, but is it really effective in practice?

Public Conversations

Good Things Here...

2007-07-31T04:41:00.001-07:00

Technorati tags: Malware, maintenance OS

Some folks get it...

http://blogs.technet.com/secguide/archive/2007/07/12/malware-removal-starter-kit.aspx

Using the Windows Preinstallation Environment (Windows PE) in combination with free anti-malware programs, the kit provides you with a low-cost, effective strategy and tool recommendations that you can use to vanquish malware attacks

...while some folks don't:

http://blogs.msdn.com/rflaming/archive/2006/09/20/763960.aspx

From a security perspective, when you get owned running under a Machine-wide account, game is over and you have to flatten the machine to get back to a secure state.

By "it", I mean the defense-in-depth concept that the battle doesn't end when malware gets into your PC. Machines get "owned" all the time; the majority of spam is carried by botnets running on such systems, and surveys have indicated a high percentage of PCs are running malware.

If the only option for such systems is to "just" flatten and rebuild, many consumers will simply shrug and prefer to stay infected. After all, they tolerate rootkits dropped from audio CDs, DoS (activation) payloads built into their OS, adverts from all over the place, etc. so why should they mind if a smidgen of bandwidth is used to DDoS unpopular entities such as the RIAA, or send out the same spam they get every day, either way?

The problem with "just wipe and rebuild" is not the pessimism that a cleaned PC will really be clean, but the optimism that a rebuilt PC will stay clean. In reality, both approaches are complex battles that may be lost.

Security Guides Blog

The first link is from SecGuide, who may be the first Microsoft team to offer end users the tools they need to formally manage malware on infected PCs. They may not be as far down that road as some Bart-based solutions, of which an example is shown in this slide show, but in-house Bart projects are usually too complex to be offered as an off-the-peg solution for end users to download and use.

The SecGuide approach is based on WinPE 2.0, which is now available for end users via the WAIK. The process of integrating tools into WinPE, and building a WinPE boot disk, is pretty daunting, so I was wondering if combining David Lipman's Multi-AV tool with Bart PE would be easier?

In the big picture, we need to market the clean state against the accepted state of living with resident malware. A non-destructive cleaning approach is a key element, and it's good to see parts of Microsoft getting this.

Windows Installer

The second link is from Setup Sense and Sensibility, which is a fascinating insight into the Windows Installer and how this has developed in Vista in particular. The perspective appears to be 100% rooted in the concerns of corporate networking, and centered on per-user permissions and control.

The trouble is, this approach just doesn't fit the outside world of free users and the one or few PCs they use. There's no "admin" to "do things for" the user; no tight white-list of permitted applications, and the user should have full and unfettered control over the PC. A single PC may represent the user's entire infrastructure, so there's no "easy way out" of wiping and rebuilding desktop systems while data is safe on the server.

Moreover, the same user will do multiple different things in the same logon session that should have differentiated rights. Simply giving all processes the same rights just because they occur in the same logon session is next to useless, as even the most limited user account rights will allow the user's data to be edited, overwritten or trashed.

I've covered aspects of this issue many times, such as the adverse effect of flattening natural obstacles and the janitor account concept. UAC is a step in the right direction, as for the first time it leverages the user's control over automated processes - the reason it is so "ugly" is because it is so at odds with the assumptions underlying NT's development, i.e. that automation would always be done by "proper" entities and that the user should be swept aside to facilitate such automation.

Timeless Blogging

2007-07-31T03:31:00.001-07:00

Technorati tags: Blogging

Can you have your blog and web it too? I'd say so, and am doing that - blogging so that content is navigable in ways more like a web site, as described here. My tactics are:

Lists as the "top level"
Permalinks, labels and tags
Hover-tips to explain link destinations
Closure via post-terminating "home" links

Blogging is still dominated by the timeline concept, and this is particularly ironic when a post solicits feedback, only to tell you that "new comments are disabled", as in this case. Bah! So I'll feedback here...

Yes, I'd love to see the SecGuide blog post discussion topics for feedback, as well as their usual announcement posts. They can use tags to separate such traffic for those who only want to read announcements, vs. those who want to contribute to discussions, and still have a mix of both for the rest of us interested in both types of posts.

I plan to do something similar on my blogs, so that in effect each blog can function as multiple sites. Blog-based discussions may be more survivable and discoverable than web or news forums, as the blog (or web site) provides a persistent navigation tree to reach this material.

Meta-Bug: UI Refresh That Doesn't

2007-07-19T05:54:00.001-07:00

Technorati tags: Bug, Meta-bug

If you can read this post, you're lucky, because I think I've just edited this blog's framework into the ground!

The case was a meta-bug (i.e. a conceptual bug that underlies other bugs). I was trying to add a poll to the blog, which was rejected due to some date format thing. So; is "07/07/19" yy/mm/dd or mm/yy/dd? Do my preferences for dd/mm/yy(yy) have any bearing on what the error report shows? None of that is the bug, by the way; just bad error message design - in fact, it may be that the new Poll thing is just plain broken.

Here's the bug: If I try a different date that is also "wrong", will I be able to visually tell the difference between nothing happening when I click Save, and getting the exact same error message invisibly "refreshed" over the old one? What happens if I ASSume that "nothing happened" and I machine-gun the dumb-ass Save button 100 times out of frustration?

Er... I wish the last question was rhetorical, heh. I've just closed the browser window that has been "Saving..." my blog for the last unfeasibly large number of minutes.

So if you're looking at a blog layout that is currently less than fully-assed, with a dead poll that was "fished" before it was born, then you know why.

Bah! User failure strikes again...

Malware - Is That All You Ever Think About?

2007-07-19T02:09:00.001-07:00

Technorati tags: Malware

Folks could be forgiven for asking:

Why do you care
About malware?

Malware is the bulk of a larger problem which is vendor-pushed code. Nothing can overwhelm support resources as widespread automatic insertion of bad code can do.

For in-house system administrators, it's a major headache, but for a tech servicing multiple single-PC sites, it can be a disaster. If you offer an SLA (Service Level Agreement) that is insufficiently escaped by weasel-wording and disclaimers, then one big outbreak can put you out of business... how do you "resolve within 48 hours" when you have 100 sites per tech needing urgent attention within the same hour?

So yes; just as someone interested in completing university studies may switch to soldiery as driven by self-preservation demands, so I have an interest in malware. And just as a soldier has an interested in keeping his weapons in working order, I have an interest in maintenance OSs such as Bart and WinPE 2.0, as well as the politics that keep these tools out of the hands of those who need them most.

Sharp readers will have noticed my definition of the "larger problem" encompasses automatic OS and antivirus updates, various ad-hoc "update" facilities built into arbitrary programs, Google's "update everything" tool, and codecs "needed" to play arbitrary content.

All of these break best-practice rules on code changes:

Do not allow others to change your code
Log all code changes
Ensure all changes are reversible
Ensure changes do not "kick away the ladder"

In essence, the logic behind "code of the day" is broken:

When our code breaks, it can't be trusted
This happens too often to manage manually
So trust us to push more code whenever we see fit

Does not compute. Yes, I see the need to patch OS and exposed surfaces as soon as possible, but I also see the need to reduce exposed surfaces made of code that is not trivial enough to be relied on as defect-free.

And no, I don't recommend Google's "update everything" tool.

New Content From Here

2007-07-16T09:33:00.001-07:00

I've just completed a wish list of around 30 or so I'd like to see fixed, changed or created in Vista (and in some cases, retro-fitted to XP).

I also did a commented picture show of a typical Bart mOS session, stepping though verifying each safety level before reaching for the next, as outlined in this PC Crisis article.

I've always wanted to re-use blog posts as structured web pages, so as to combine the simple creation and consistent style of a blog site with the ongoing re-usability of a formal web site. I find I can do that on the other blog by combining the "list" feature with jump-pad blog posts full of links. I'm used to this structure from a closed mOS wiki I did a while back.

What this may mean, is less (or perhaps, different) content here, with links from here to there. I like the lack of adverts on this site, but I can't see as easy a way to get it to do what I'd like done.

Forthcoming Attractions

2007-07-13T05:55:00.001-07:00

In the next week, I'll be low-profile in newsgroups (again) and doing Vista feedback and mOS issues that I will echo in my blogs. The mOS stuff will be here, while the Vista stuff I'll do over in the Vista Curve blog.

I expect there will be a lot of traffic, so suggest you peruse subject lines in the sidebar rather than desperately hitting the PageDown key :-)

mOS for Vista: WinPE, WinRE and Bart

2007-02-11T00:36:00.000-08:00

With Vista, one loses some friends and gains others, with Vista-64 being particularly challenging. You can use XP-based Bart to do many things in Vista (especially hardware diagnostics) but nothing has the ability to redirect registry access on the fly as RunScanner does for XP on Bart, which is a significant limitation for Vista maintenance.

The players are:

MS WinPE 2.0 for XP, 2003 and Vista
MS WinRE for XP, 2003 and Vista
Bart PE, current XP/2003 version
Bart 4 for Vista, XP, 2003 etc.?

Bart 4 is but a rumor at this time, and may stay that way awhile. As MS WinPE 2.0 is now available to general users, and as Bart has experience with earlier MS WinPE, he may develop a GUI front-end for that instead (I'm just guessing here, but that's what I'd do).

My Vista Curve blog will cover WAIK, WinPE 2.0 etc. as I grow my Vista maintenance skills. There are also walk-thoughs for building MS WinPE 2.0 solutions, and blogs from the Microsoft WinRE and WinPE teams.

I was going to develop MS WinPE 2.0 as a separate mOS project, but a better way might be to cross-develop Bart PE to be used as an extension of the MS WinPE environment. This is suitable for techs with an \i386 file set, but is tricky otherwise. On the other hand one could easily build a gutted Bart with no bootability, no OS, and just the nu2menu interface to collected tools.

MS WinPE 2.0 seems harder to build and use than Bart PE. The Bart equivalent of those 4-page walk-throughs is "install Bart PE builder, run it, and follow the UI that pops up". Once you have the skills, similar things should be possible in MS WinPE 2.0, but those who find the Bart skills investment too high will bounce off WinPE.

MS WinPE runs in RAM, allows the boot CDR to be removed, and allows arbitrary CDs and USB sticks to be hot-swapped - so the potential is there to launch your Bart CDR from a MS WinPE boot via an accessible "pull tab". On the Bart side, you'd either have a Vista-specific nu2menu, or logic within the menu and/or plugins that filters and modifies things to work (or not be offered) in Vista.

My Bart project is already down this road, built for compatibility with...

Running from output subtree on the build PC
As accessed from DOS diskette boot
Autorun from Windows 2000, XP
Autorun from Windows 9x
Booted into Bart mOS

...so adding "run from MS WinPE boot" is just an extra step.

You can already manually run a Bart PE CDR's menu from an MS WinPE boot session. Some tools don't work as needed code files or registry settings aren't present in the WinPE runtime, and that may be a ball-breaker to fix. Some tools don't run due to permissions issues, and may run if it were possible to do the equivalent of right-click, Run as Administrator within the WinPE environment. Others work just fine.

The nu2menu system has some breakage when run from an MS WinPE 2.0 boot. So far, one issue I've picked up is that the very useful GUI file/folder selector feature fails to populate the namespace tree, so you can't use that to set Temp etc.

As an MS WinPE builder, you'd likely have or will soon build skills in XML. The nu2menu system is XML-based, and could be used in two ways (as I do in Bart):

UI front-end for the build process
UI front-end or shell for the mOS runtime

The MS WinPE build process would benefit greatly from some sort of GUI front end, and nu2menu would be useful for that. It runs as an unobtrusive button in one corner of the screen, and from there it fans out like the traditional Win9x Start menu, but with a lot more power and logic.

HD Replacement and Bad Cluster Markers

2007-02-05T11:45:00.000-08:00

This is a common scenario...

Hard drive develops bad sectors
File systems have some clusters marked as Bad
You image partition(s) from old hard drive to new replacement
New hard inherits bad cluster markers that are now inappropriate

...that can present in interesting ways, if file system is NTFS:

BING images the sick hard drive OK
BING images the contents to new hard drive OK
You use BING to shrink the partition
BING checks the file system OK
BING shrinks the file system OK
BING re-checks the file system, and fails!

The FATxx equivalent is more like this:

BING images the sick hard drive OK
BING images the contents to new hard drive OK
You use BING to shrink the partition
BING checks the file system, fails!
You manually clear bad cluster markers
You use BING to shrink the partition
BING checks the file system OK
BING shrinks the file system OK
BING re-checks the file system OK

What is going on here? Well, it depends on the file system.

With FATxx, BING sees the bad cluster markers when it checks the file system before it attempts the resize, "sees" the bad cluster markers inherited when the file system was imaged off the old hard drive, and backs out of the resize. Only when you clear these bad cluster markers (as you should) will BING attempt the resize, and will then generally succeed.

With NTFS, BING fails to see the bad cluster markers and is happy to resize the volume, which is dangerously inappropriate if the volume is still on the sick hard drive. NTFS points to bad clusters by cluster address; if these point beyond the end of the file system after the partition is shrunk, they are seen as invalid by BING's post-resize file system check.

Clearing Bad Cluster markers in FATxx

If there's a better way, I haven't found it! What I do is edit the file system on the new hard drive using ye olde Norton Disk Edit, selecting the two FATs in different panes, searching these for the raw byte sequence that marks a bad cluster, and editing these to 0 (free cluster available for use). When done, a file system check can be expected to detect FAT32's boot record free space value to be incorrect, and I allow that to be fixed.

Obviously this is serious caveat territory, and I look up FAT details to see what the Bad Cluster value is, and then run into byte-order toe-stubbers while searching for these. Norton Disk Edit runs from DOS or a Win9x DOS mode, and DOS can't safely "see" beyond 137G, so more caveats there.

Clearing Bad Cluster markers in NTFS

I haven't found an easy way to do this, but the process is documented at Katy's Homepage - so if you know of any free tools that can automate this, please comment with details!

If the Bad Cluster pointer is invalidated by shrinking the NTFS volume or partition, as described above, then the invalidated pointer will be "fixed" (cleared?) by ChkDsk /F

The problem is those Bad Cluster records that remain valid, and thus are not cleaned up by ChkDsk /F, or (if marked clusters are not re-tested) by ChkDsk /R.

Blogger! Can't Update Site Links!

2007-01-28T09:43:00.000-08:00

When I first set up this blog, I entered links to my main web site and older Win9x web site, while leaving in the Google News link that was there.

Hmm... according to Google News, "Palestinian Leaders Plead for Clam"... not clear if they're asking for clam to be served them as food, or perhaps clamency for an aquatic comrade in arms?

Anyway, what I wanted to do was add my new Vista blog, but I can't see anywhere to edit that list of links - and I did look everywhere. Strange!

Internet Explorer 7 Exits When Started?

2007-01-28T09:14:00.000-08:00

I see Microsoft has an article describing this behavior, though in my experience the problem may be more general than the article suggests, though the mechanism may be the same.

What happens is that whenever you run Internet Explorer 7 (IE 7), or things that invoke it such as Windows or Microsoft Update, it vanishes off the screen as soon as it appears, with no error messages as to why it has done so.

I've seen this in previously-working IE 7 installations, but more commonly after something has interrupted an IE7 installation. What would do that? Automatic Update, that's what... as seen in the "Windows Bugs" photo set at my other blog.

I've seen this regularly, as others have not, so I wondered why this might be. Perhaps it's because the PCs in question have a lot of old patches to catch up on, so Automatic Update gets active as soon as they reach the Internet, plus I often install IE 7 from a saved copy off CDR at around the same time.

What is supposed to happen is that the Internet Explorer 7 install does its thing, then prompts you to restart Windows within its own series of successive blue dialog boxes. Instead, these dialogs are still indicating files being installed etc. while Automatic Updates pops up its usual grey dialog telling you to restart Windows, and if you cancel that, it will pop up again and again.

I've always wondered whether Automatic Update co-ordinates itself with what Windows or Microsoft Update are doing, or whether the same material gets downloaded by each at the same time, doubling the bandwidth consumed. This case suggests problems of that nature; the IE 7 installer should trap and disallow (or gracefully clean up) software-initiated shutdown requests, and/or prevent other items installing themselves while the IE 7 install is in progress. Similarly, Automatic Updates should detect Microsoft's own installation activity, be this locally or as managed from update web sites.

Sun Java JRE Bloat

2007-01-28T08:55:00.000-08:00

Well, it took Microsoft long enough to finally scale down Internet Explorer's ridiculously bloated cache allocation; Internet Explorer 7 follows other browsers in sizing this to 50M, irrespective of hard drive volume size, and it may complain that the present cache size is too large (e.g. if it was set so via user).

However, what Microsoft has finally learned, Sun is still getting wrong. After installing the new Sun Java JRE 1.6, I saw a Java SysTray icon, and poked around; there's a slider for temporary file cache to be allocated to Java (separate from browser caches) and the duhfault is 1G! Needless to say, that got scaled back to 20M pretty quickly.

So, there's a new bloat factor to remember when folks run out of space on C:, over and above the bloat of multiple Sun Java JREs, as discussed earlier in this blog. At least new JREs no longer pass control to older (exploitable) ones when requested to do so by Java malware (sorry, "valued Java applets"); still, at 100M+ a pop, old JREs aren't too cheap.

Bad RAM, Bad RAM Tester Design

2007-01-13T15:43:00.000-08:00

This long post covers Vista's mOS, MemTest86 and Microsoft's stand-alone RAM testing utility.

How bad RAM presents

If RAM was originally OK, then goes bad, you'd start to see random errors, crashes, lockups, reports of corrupted registry or other files and operations, and perhaps some spontaneous resets. This random pattern may develop some reproducible errors, where the contents of the hard drive have been corrupted, either from bad RAM per se or from recurrent bad exits.

RAM crashes at full speed, so you won't notice any slowdown of the system. This contrasts with bad sectors on the hard drive, which slows the system due to attempts to retry the operation, and/or copy contents of failing sectors to spare sectors. On most consumer PCs, there's no attempt to detect RAM errors after the BIOS boot phase; where such detection is present, the system will usually halt as soon as a RAM error is detected.

Why bad RAM matters

RAM errors can not only corrupt what is written to disk, but also where it is written to disk, at a level beneath that of the file system. A sector intended to be written to the contents of a file may instead be written over some core file system structure, e.g. if a hi-order bit in the raw sector address is flipped from 1 to 0.

RAM errors can corrupt code, causing crashes, but a greater risk may arise where the code does not crash. Many disk operation calls may use a status byte in a register to differentiate between read and write operations, so a bit-flip there could cause a write instead of a read. This is why no disk access can be considered safe; any disk access starts with reading crucial areas of the file system, and if those reads become writes, the disk contents could be trashed.

Why bad RAM may be tough to exclude

When I started out with PCs in the era of DOS 3.3, 286 processors etc. I wondered why there were so many RAM testing utilities around. Surely you would just copy data to a spare register, write it into RAM, read it back from RAM, and compare with the spare register?

I had found that in practice, several testers would pass RAM as "OK" even though swap testing would clearly demonstrate that problems would clear up on the suspect PC with good RAM and appear on a known-good PC with the suspect RAM added.

So I though a bit more about how RAM can fail; not just by returning different data compared what was written to it, but altering data in other addresses when certain addresses are accessed, or behaving differently according to whether the RAM is read for instructions vs. data, or whether it is being accessed by the processor, AGP, or some other device via DMA.

Also, some failures can crash, lock up or reset the system, instead of being presented as a nice list of bad addresses. If the RAM testing boot disk is left in the system during the test, a spontaneous reset may be missed, unless you happen to notice the test has been running for fewer hours than have elapsed since you started the test.

For a long time, I gave up on RAM testing utilities, and just did swap testing as above. My faith in RAM testers started to return with SIMM Tester, and grew stronger with MemTest86 and MemTest86+. But I find that even with these tools, either one of the two MemTest86 projects may detect errors where SIMM Tester does not, and 8 hours of MemTest86 may pass, only to throw errors somewhere in the next 12 hours of testing.

How to design a RAM testing utility

This isn't about test sequence and data intended to provoke errors due to local power starvation or whatever. Instead, it's about how this core of test routines should be wrapped into a safe and usable utility - as illuminated by issues raised earlier in this post.

Microsoft have a free stand-alone RAM tester that is called the "Windows Memory Diagnostic". But why is "Windows" in the tool's name, given this is a tool that should run at the system level, before any OS has booted up or is left running in the background?

I used this stand-alone form of the tool, and noticed something rather nasty about it - when set to repeat the test sequence, it clears the results of all previous test passes! It also does not indicate elapsed clock time, so if the tester disk is left in the boot drive, the test will restart and look exactly the same as if it had been running without any interruptions.

Any RAM failure is significant, even if it shows up only once in 24 hours of testing. If you use MemTest86 and one such error occurs, you will see it listed when you return after an overnight unattended test - whereas even if Microsoft's tester flagged it at the time, you will only see the "OK" result of the last test pass when you return in the morning.

There's no point in doing 24 hours of testing, if only the last pass (possibly the last 20 minutes of testing) is reported! Who is going to sit and watch an "unattended" RAM test loop for 24 hours, just in case one pass fleetingly shows an error on screen?

How to integrate RAM testing with a mOS

I'd love to include RAM testing within my maintenance OS, but I can't see a way to fully automate this. The mOS boot disk should not boot past the RAM testing component into loading the full mOS, because that involves a lot of disk operations that may be unsafe when RAM is bad. There's no safe and standard way that the RAM tester can set a flag that it is in session, that will persist after a spontaneous reset. The best I can think of would be to boot the mOS to a menu that defaults to testing RAM, but that does not timeout but will wait forever for a keypress.

So I can't see a safe way to incorporate RAM testing into a wizard-driven mOS intended for unskilled use. It would be lovely to have a boot disk that would do x hours of RAM testing, then test the hard drive for physical errors, then test and possibly fix file system logical errors, before commencing with formal scanning for malware. But without a safe way for the mOS boot to detect whether RAM had been recently (define "recently") tested without errors, the best I could design would be a mOS that booted to a 3-item menu (test RAM, continue with the wizard, or display help) and stayed there until a selection was made.

How to get all this sooo wrong

The good news is that the Vista DVD has RAM testing incorporated into the mOS. The bad news is that Microsoft made just about every mOS design mistake possible:

mOS boot will fall through to hard drive boot unless key is pressed
mOS runs a lot of code before the UI from which RAM can be tested
mOS looks for a Vista installation on hard drive before anything else
mOS drops RAM tester on hard drive, then reboots to run it
RAM tester does one pass only, unless this is overridden by user
RAM tester displays no results on screen
RAM tester writes results to Vista installation's logs on hard drive

OK, let's walk through what happens when you test a system that may have bad RAM. Microsoft seems to expect this RAM to be so bad that a test single pass will catch it, even though we know from experience that you may only see one error in 24 hours of testing (mistake 5).

If RAM is so bad that one test pass will always catch it, then it is surely too dangerous to run large complex GUI code (mistake 2), or to read into the logic of a Vista installation on the hard drive (mistake 3). If BIOS standard practice is to halt a system whenever bad RAM is detected, irrespective of what the OS was doing at the time, then surely it is foolhardy to boot up a complex OS from the hard drive (mistake 1), write material to hard drive (mistake 4), especially if the RAM has been proven to be bad (mistake 7)?

What happens if the nature of the defective hardware causes the system to reset, rather than lock up or carry on running so the tester can flag the error? Well, the Vista DVD will chain into the Vista installation on the hard drive and boot that (mistake 1), which is about the worst possible thing one can do - and this will happen even if you had explicitly excluded the hard drive from BIOS bootability, because the DVD boot chains directly into it irrespective of any BIOS-level settings you may have applied.

If the RAM did test bad, how would you know? It seems as if the only way would be by booting Vista from the hard drive and scratching around in Event Viewer. If the process of writing those results into Vista's logs didn't corrupt the contents of the hard drive, then booting Vista (with all the attendant paging, temp files and registry updates this may imply) to reach Event Viewer may well do so.

This is a bit like being a driving license tester faced with a pupil who immediately tries to mash down pedestrians a la Carmageddon at the start of the test. It's nice to see Microsoft (at last!) taking an interest in maintaining sick systems, but the lack of insight displayed is scary.

Learning Vista

2007-01-13T15:26:00.000-08:00

See http://cquirke.spaces.live.com, which is where I'll blog my initial bewilderment and hopefully progress as I actually work with (as opposed to, look at) Vista.

Fair-Weather Scanners

2006-12-31T05:47:00.000-08:00

I've used a few on-demand antivirus scanners and scanners for commercial malware (usually known as "anti-spyware") and generally they're just not designed for troubleshooting environments such as Safe Mode and Bart CDR boot.

Fancy display mode required

Common advice is to use these scanners from Safe Mode, where screen resolution is usually low (say, 640 x 480) and color depth is low, too (typically 16 colors).

A Squared is almost unusable in low res, because the dialog boxes ASSume you have at least 800 x 600 to play with - often the UI controls are below the edge of the display when in Safe Mode, so you have to guess the number of times to press Tab in order to keyboard the "go" button. The need for this high resolution has nothing to do with the amount of content that needs to be displayed on the screen, and everything to do with wasteful eye-candy UI design.

AdAware delights in using subtle colors that turn to stippled mud in Safe Mode's low color depth, and some needed UI cues (e.g. which UI control is selected) vanish completely.

Mouse required

Both AdAware and Spybot border on the unusable when a mouse is not present, as may be the case in troubleshooting conditions. Freshly-installed Spybot starts with a set of "wizard" dialogs that defy attempts to switch focus from the keyboard, and AdAware's keyboard navigation is highly ambiguous at best.

Installation required

The free BitDefender 8 on-demand scanner and MS Antispyware (now Windows Defender) both require Windows Installer to install, and that service is not present in Safe Mode. In order to use these tools, you first have to run normal Windows - so that the malware you are after is almost certain to be active and well-positioned to interfere with the installation and use of the scanners.

I haven't yet got the above tools, or AVG Antispyware (ex-Ewido), to run from a Bart CDR boot. Trend SysClean, A Squared, AdAware and Spybot are better there, with Spybot claiming the ability to scan relative to the inactive hard drive registry hives without needing RunScanner redirection. In practice, I find Spybot detects less when run from a Bart CDR boot than when it is run from Safe Mode.

Vista vs. email

2006-12-21T00:56:00.000-08:00

This blog post was interesting:

http://windowsvistablog.com/blogs/windowsvista/archive/2006/12/19/windows-vista-and-protection-from-malware.aspx#comments

It's an interesting expectation, that Vista would magically be immune to malware attacks - but that expectation is taken seriously in this post, which views the problem through the eyes of the totally inexperienced user. By blocking access to all incoming attachments, Vista's native Windows Mail is able to foil 8 of the 10 common attacks tested - the ones that got through, did so by using file types that some email applications don't block.

My expectations are far more modest:

System should be immune from clickless attack
User should receive accurate risk information
System should act within the bounds of that risk information
Should malware go active, user should be able to clean it

Jim's assessment treats the user as a passive component that has to be protected by the system acting on the user's behalf. I see that as unrealistic, and not only because it's a "mission impossible" task, but also because most users will disable total attachment blocking and then lose that degree of "protection". So what works for great disclaimable advertising copy - "used as directed, Windows Vista is immune to 8 out of 10 common email attacks" - works less well when users actually use the system to do real-world things.

For many (most?) users, blocking all attachments is too broad a sword to live with. What these users expect, is to look at an email message and attachment link, and assess whether the attachment is safe to "open". That in turn requires information about the attachment file type that is easy to understand (as a large number of raw .ext is not) and can be relied upon (in contrast to Vista's default "open based on hidden info rather than visible .ext" behavior).

Windows has been designed with many things in mind, but type discipline is not one of them. There's been great stress on per-user rights in NT, in keeping with the needs of corporate IT, but this maps poorly to consumer needs. The code/data distinction has been undermined, and the unrealistic objective of "you can do everything without having to know anything" assumes that consumers won't have the skills to assess and act upon file type risk information.

The last point, "should malware go active, user should be able to clean it", is a topic in itself, which goes about safety awarenss that stretches from maintenance OS through "Safe Mode" and into safe handling for suspect locations, such as newly-discovered drives or subtrees that are designated as holding risky material, much as "My Documents" is designated as holding "user data".

Here are a couple of unrelated quick things...

Screening spam

Another thing I'd like to see in an email application is better filtering, based on criteria other than various text matches. Specifically, I'd like to filter out "messages" that have under 100 characters of visible message text plus embedded (or remote) images. This is emerging as a common form of spam, with two effects; firstly, there's no text to filter/match, and secondly, the entire "message text" can be one huge clickable surface.

Firefox's killer feature

Spell checking within text edit fields - a must-have, in an age of online text composition e.g. blogging, forum posts, comments and web mail!

Up until now, Microsoft has positioned spell checking as part of MS Office, with the unique vendor advantage of integrating this application component into the OS (e.g. Outlook Express).

These happy days should be over, thanks to Firefox 2, just as free Google email killed the acceptability of the 1-2M email storage norm for paid-for ISP email "services".

Bart vs. BAD_POOL_CALLER

2006-10-16T17:53:00.000-07:00

BAD_POOL_CALLER is one of those scary STOP errors that one may see in XP (that is, once you kill the duhfault "restart on system errors" setting) . This particular case was an XP SP2 system that was said to crash straight into this on startup.

Uneventful 12 hours in MemTest86 with no sponaneous lockups or booting of the Bart CDR I left in place during the test, motherboard caps OK. Bart CDR boot, HD Tune passes SMART, temperature and surface on both hard drives, file systems OK on ChkDsk all volumes.

Formal malware scans fine, until the first test that requires RunScanner to access the registry hives on the hard drive. As soon as I click RunScanner's dialog OK, the system STOPs.

Riiiight... next, I harvest spare registry hives from SR Restore Points in the C:\SVI subtree. Then I pick a trivial scanner that I've set to run via RunScanner; in this case, Stinger. It doesn't matter what it is; all I want to test-to-fix is whether I can initiate registry access to the hard drive hives. If that no longer dies, I may have fixed the problem.

I run Stinger in this way, each time choosing a different user account and not checking the "use all hives" checkbox in the RunScanner dialog box. Ever user account is fine except the one they actually use, which dies the blue death.

Now that I've narrowed it down to a single file, I rename away that user account's NTUSER.DAT (which is the per-user registry hive), copy in the most recent spare from the most recent Restore Point, rename that into action as NTUSER.DAT, and re-test; this time it works as well as the other accounts did.

I'm interested in a single hive causing this common head-scratching problem, so I keep the "bad" and "fixed" copies of the hive, which appear to be the same length. I'll FC them to see if there's some specific difference (either structural, or a recent install... tho I'd expect the latter to change the file length) that causes the problem, and update this article if that looks interesting.

Meantime, Bart has saved the day yet again; what could so easily been "just" wipe and re-install, turned out to be a few unattended hours on "the prelim" plus around one hour of interactive work in Bart. Did I mention I liked Bart?

That drill-down method again...

check RAM and hardware first
Boot into Bart CDR
use a tool via RunScanner
choose one user account at a time
if all break, suspect a system hive (common to all accounts)
if only one account breaks, it's that account's hive
preserve damaged hive by renaming away, not delete
harvest replacement hives from recent Restore Points
try with newset, then second-newest etc.
do not try any of the above in hard-drive-booted Windows
compare bad and good copies of the hive for differences

There are probably scores of reasons to STOP on BAD_POOL_CALLER, and many of these may have no pattern if the underlying hardware level of abstraction is bad (as is often the case with registry damage). Even so, if you have a consistent STOP on every boot, at the same point in the boot, then this approach may find the solution if your case is like this one.

Open Source Eudora

2006-10-15T01:07:00.000-07:00

Most of the time you'd be reading rants about awful and shifty vendors are - perhaps every industry is as bad, but I'm "further away" from most? - so it's a pleasure to celebrate software vendors who do the right thing...

http://www.eudora.com/faq/

Now here's a vendor with a popular product, but one that isn't their central interest. They could have just killed it, and told those who complain that it is their right to do so; after all, we see that all the time with music corporations, who delete titles they are "too old and unpopular to make money" while still forbidding even the original artist to distribute them for free.

But instead, they are shifting the product into Open Source, while committing to honor their obligations to those who have purchased the Paid version. Those who use Sponsored mode (myself included) can now stay in this mode with full functionality forever, even when the ads stop. Unless there's some sting in the tail so hidden I can't see it, it looks like an excellent result!

Previous "Do The Right Thing" award

I've been as impressed a few times before, and the last time was when Computer Associates ceased the popular free InoculateIT antivirus suite. Again, they announced the move and then supported the free version with updates for a longer period than commercial vendors' one-year subscription, and they offered a low conversion price to the feeware eTrust that replaced it.

The InoculateIT story was particularly impressive, as the initial announcement that stated "free updates until we have to change the scanning engine code" was made within a few months of the release of a new version of Windows. It would have been so easy to claim the need to create a new engine to overcome compatibility issues with the new Windows version, but they didn't do so; InoculateIT remained free for many months thereafter.

When InoculateIT ceased to be free, it also ceased to be the de facto free/non-warez antivirus product. AVG stepped into those shoes; it was always around, along with Avast, later AntiVir, and some others, but there was greater confidence in InoculateIT at that time. AVG have also done the right thing when they dropped the free AVG 6 product to consolidate on AVG 7 as the sole code base; they offered a free version of AVG 7, pushing alerts to AVG 6 installations about the cut-off date for some months before updates ceased for the old version.

When you have such good "no strings attached" free antivirus products, why would anyone want to put up with Symantec's embedded commercial malware in Norton AV? If you do a Google( "Why I don't use Norton" ), you will see I'm not the only one who avoids it.

Rungbu.A Exploits Bad Design

2006-10-14T16:16:00.000-07:00

This case study illustrates several issues I've raised before, as well as a few lessons, such as "there's no 'one problem per case' rule", "best practice isn't bullet-proof" and "one antivirus scanner isn't enough".

I was on site doing something else, when I was called to check out a problem with opening Word documents, which the user attributed to an encounter with a dubious diskette.

The first thing I noticed was that her PC wasn't showing file name extensions, contrary to the way I generally set up PCs...

"Hey, you can't see the file name extensions! Without that, you don't know what type of file you're about to open! That's dangerous!"

'No, that's OK; I can see the Word icon, so I know the files are Word documents'

This was followed by an explanation of why this can't be trusted, while she insisted it was OK, and 'was always like that'. I pointed to two files as an example; a pale (normally hidden) one called "Some file name" and a bold one also called "Some file name". I right-clicked on each, and sure enough, the hidden one was the .DOC while the visible one was an .SCR - so I wasn't too surprised when the setting to not hide file name extensions would not "stick".

"You're malwared", I said, and after shutting down and setting CMOS to boot CD, I booted up the Bart CDRW I tend to have on me at all times. Bart would boot on this crusty old Win98SE system (333MHz, 64M RAM)... if only the 32-speed CD-ROM would read CDRW disks... so it's heigh-ho, back to base we go.

Who's stupid?

As a geek, my first reaction was to consider the user foolish for trusting icons as an indication of file type. Then I thought; why should a user know that the most dangerous file types can set whatever icon they like, and that .scr files are raw code, and thus dangerous? Why doesn't the user interface clearly flag which files are code and which are data, as well as the type, and disallow any content to misrepresent itself? Why are file name extensions hidden by duhfault, anyway, and why are things still as brain-dead in Vista?

That's the problem with bad design - it never gets patched, because it "works as designed". We had years of MS Office macro and VB malware before that was fixed, years of Outlook and Outlook Express auto-running scripts in HTML "message text", and we still have Format in the middle of hard drive context menus while Backup, Check for errors etc. are buried under Properties, Tools. Stupidity is found not only in end users.

Best Practice can still fail

As usual, I started work on the system with several hours in MemTest86, to make sure the system was safe to run at all. Then I booted my Bart CDR, eyeballed SMART details in HD Tune, did a surface scan of the 4G hard drive, and checked SMART details again; no change in SMART, no surface errors, OK. A ChkDsk confirmed the four file systems were OK, so I created a session base directory on one volume, and set that as Bart's Temp location. I could then shrink the Bart RAM disk as it's no longer needed for Temp, and create a pagefile on hard drive to relieve constraints imposed by 64M of RAM.

Then I started my antivirus scanning wizard and went about my other work. A while later, I see the second av scan is still stuck on the same file, so I run HD Tune again; it shows blank SMART details, and a surface scan picks up "one bad sector".

I immediately pull the mains, pull the hard drive into another PC, copy off everything from DOS mode using the LCopy from Odi's LFN Tools, starting with the data set and carrying on until most stuff is backed up. I had hard failures on C:\Windows (bad disk) and the session subtree to which the Bart av wizard would have been logging the scans (file system corruption).

Next, I went in with DiskEdit, confirming bad clusters throughout the entire C:\Windows directory chain. Noting the cluster address of the Windows directory, I searched for subdirectories on C: (fortunately it's a small C:, not the whole hard drive) and ballpointed the . cluster addresses for all that had .. pointing back to the lost Windows directory. Then I created scratch directory entries in C:\ to point to these, and copied them off.

I then did a raw image copy of the fortunately-small C: volume in case I needed to recover more stuff later, and finally back in DiskEdit, I "erase-marked" the Windows directory so that scanners traversing the file system wouldn't fall into a pit of bad sectors.

Having got what I could off the stricken hard drive, I put it back in the PC it came from and got back to my Bart antivirus scanners etc.

Rungbu.A

Four out of five of the initial "detect only" scans detected the same files as infected, but each called the virus something different. One called it a generic trojan, another called it SillyWorm, each with a high variant suffix. Only Sophos gave it the unique name of Rungbu.A, though their site only had a descriptive page for the Rungbu.B variant. The sixth scanner was set to kill, and did; thereafter there was nothing for the remaining scanners to detect.

Reading the decription's Advanced page revealed this malware to be anything but "generic". It left the system tattoo'd so that I had to Regedit before I could stop Windows from hiding file name extensions.

We get annoyed when vendors don't patch known exploitable surfaces, and highly irate when there are ITW (in the wild) malware already exploiting those surfaces. Yet we've seen so many malware with double file name extensions such as README.TXT.pif, and these raw code file types can and do set their icons to match the faked file type.

But hey, not a problem; it "works as designed".

Re-entry

Finding the plethora of vintage application disks for this PC would not be fun, so I decided to preserve the old installation instead. I called the site and asked them to find the disks if they could, in case I'd have to rebuild, then set out to fix the installation.

First, I partitioned a replacement hard drive (a used 40G, jumpered to act as a 32G in deference to the old PC's BIOS limitations) and copied everything to one of the logical volumes. Then I fresh-installed Windows 98, and copied that subtree to the logical volume as well. Next, I copied everything except the old Windows child subtrees into place, then finally identified and copied the recovered child subtrees over what was installed with Windows.

All of this was done from DOS mode, but I couldn't extract recovered registries etc. from the latest RB*.CAB from there, so I had to go back into Windows at this point. That crashed Explorer, so I set shell=Winfile.exe in System.ini, and from there I could Extract the registry files. Back to DOS mode to drop in these registry files, as well as older backed-up Vmm32.vxd and "Exit to DOS.pif", and now everything looks OK - though I'll try everything out in case there are needed files that are missing from the Windows base directory.

Duhfaults are forever

It's a good thing the variant of Rungbu that infected this PC didn't also put "hide hidden and system files" into effect, and that we don't use Microsoft's duhfault settings. If we did, there would be no visible indication the system was malware'd; we'd see only one "Some Name" file, which would appear to "open" just fine (the malware code runs invisibly and then spawns and opens the original Word document). Unless someone tried to change the Explorer settings, and became puzzled when the changes didn't "stick", there's be no indication that anything was wrong.

And that means the companion malware files would have found their way into every data backup, too.

It's all very well saying "it's only the default setting; you can change it", but defaults are forever. These unsasfe defaults are all you get in "Safe Mode", will recur after "just" formatting and re-installing Windows, will be the baseline for every newly-created user account, may be re-asserted by domain servers or when account rights are limited, and will be what users see whenever they use arbitrary PCs elsewhere. Defaults should always be truly safe!

Vitsa's Maintenance OS

2006-10-02T11:59:00.000-07:00

This is one of the best bits of news I had from the Vista Labs a few months back! We were told to spread the word about what wasn't NDA, but this item was NDA at the time, so I had to sit on it. But today' Google( Vista boot DVD WinPE ) shows it's public knowledge now :-)

http://www.apcstart.com/site/jbannan/2006/08/1082/windows-pe-20-a-tiny-version-of-windows-for-system-maintenance

This turns what would have been a crisis (would Bart CDR boot be compatible with Vista's NTFS, registry, etc.?) into what may be a reason for cautious consumers to favor Vista over XP.

I tested Vista beta 2 a while ago in a bit more depth than recent time permits with the newer Customer Preview build I have now, and it's certainly come a long way since that earlier build. I specifically wanted to test the mOS, and that turned out to be very interesting indeed...

Those familiar with Bart PE would guess what I'd be looking for first - can it boot off a USB stick? Can you hot-swap USB flash drives? Can you use the optical drive or will the system crash if you eject the Vista boot DVD? Is there a GUI?

No, there's no GUI - it's more like Safe Mode Cmd Only, which is a good thing in many ways. I'd have been worried in Explorer was there as the shell, in case Vista's richer shell offered exploit surfaces to malware on the maintained system.

Yes, you can eject the boot DVD! In this recent build I tested, the Vista installation DVD is the mOS boot disk, and just as you'd UI your way to Recovery Console after booting an XP CD, so it is that you GUI your way to "command prompt", which is likely to be WinPE 2.0 itself.

After booting, the DVD gets a different drive letter, compared to the booted OS files. The free space and a few other cursory tests indicated these were different volumes, and neither is an alias of hard drive space. You can eject the Vista DVD, insert other CDRs or DVDRs, and use them directly. I suspect the mOS runs from a RAM drive - and it worked quite happily in 512M RAM. What I didn't check was whether it uses a page file on the HD.

Vista development takes off from the most recent Server 2003 SP1 code base - and this is a good code base for a mOS, because it no longer resets the USB during the boot process, as XP SP2 does. So the odds are favorable for booting Vista mOS off USB flash drives, etc.

Unlike a Bart boot, Vista mOS will "see" USB flash drives inserted and changed on the fly - they don't have to be present at boot time, as they do with Bart, and swapping them is OK too. (Tip for Bart users; a memory card reader present at boot will generally allow hot-swapping of cards after boot - so I share SD cards between Bart sessions and my camera, instead of using slower and more write-limited flash drives).

Running tools from Bart CDR

Just for laffs, I ejected the Vista DVD and popped in my Bart CDR. The nu2menu (the standard "Start buttom" menu shell for Bart) worked fine, and many of the tools worked too. Because the Bart drive letter is not the same as the boot drive letter, my own "Is this booted or Autorun?" batch file logic, e.g....

Set Prog=Ad-Aware.exe
Set Launch=%~dp0..\RunScanner\RunScanner.exe
Set Opt=/m /t 0

...

If "%SystemDrive%"=="%~d0" (
Start %Launch% %Opt% %~dp0%Prog%
) Else (
Start %~dp0%Prog%
)

...concludes Bart is being run from the "native" system OS, and not as the booted OS.

That means my tools weren't running through RunScanner, which is probably prudent at this stage. Yes, that means registry-orientated tools such as AdAware or HiJackThis will not "see" the HD installation's registry, but until we know RunScanner and legacy registry access methods are compatible with Vista's registry, it's safer this way.

Many tools didn't work, because they relied on files and settings within the running OS. The Bart plugins for these tools would have included these in the Bart mOS, but that's not the OS that's in effect here - so if these resources aren't in the standard Vista code set, then the tools won't work. That's to be expected; after all, if I'd just scraped them onto Bart without using the plugin system, they wouldn't have worked there either.

All this testing was with the original Vista DVD - I haven't gone as far as building a new Vista mOS boot disk, nor have I explored "plugging in" tools as one does for Bart. I'm not sure if either of these things would be possible, or whether the answers would change between the build I tested and the final release.

Ah, for the time to really explore this stuff!

Conclusions

It's really good news to see a mOS for Vista, even if it's still not really orientated to mOS work. For example, it won't operate unless there's a visible Vista installation on the hard drive, and the RAM testing component writes to and then boots from the hard drive installation - both of which are bad practices when dealing with systems that are really ill.

I think this is because the mOS is still rooted in its origins as a "(p)re-install environment", originally intended for use on perfect fresh hardware. It was somewhat in response to this, as well as seeing some Bart off-shoots that also break some mOS best-practices, that prompted an earlier "How to design a maintenance OS" post in this blog.

The important things is that it's there, on the installation DVD (a break-through, if you'd ever peered longingly at MS WinPE though the previous licensing sphincter) and that the architecture seems fundamentally sound. It needs to be tested more rigorously to see how well it stays within the rules of mOS best practice, but it's already more than I'd dared hope for!