Category Archives: Windows XP

CHM Confusion

Recently, I downloaded the Sysprep package from Microsoft for XP SP3.  It does not really matter why.  The package was expanded and the contents explored.  However, what happened next was unexpected.  See the screen capture below:

AddressNotValid“The address is not valid” !  How baffling is that?  Not only is it not using the Internet to get the content, there is no context information describing how this happened.  My first assumption was that I did not have the same version as the content.  This turned out to be untrue.

The truth is that Windows is blocking the content since it came from another computer.  If you look at the properties of the CHM file, it will show ‘Unblock’ as an option.  Select Unblock and the problem goes away the next time you run the CHM.

If you would like a more detailed explanation, check out this post.

Cluster Map

Defragmentation on Windows XP

 

One aspect of volume management is knowing which clusters are free and which ones are used.  This is typically something managed solely by the operating system but it is sometimes possible to get a glimpse of how things align.  Microsoft published a few interfaces a few years ago that were once considered undocumented.  The set of API targets being able to defrag a disk.  The cluster map is gathered using FSCTL_GET_VOLUME_BITMAP.  A cluster is the most basic unit of the file system.  It is defined by what is specified in the boot sector of the volume.  Windows apparently always uses a sector size of 512 bytes with the option of different cluster sizes (multiples of the sectors).  The two fields in the boot sector are “sectors per cluster” and “sector size”.  The boot sector has this information at offset 0x0B for “sector size” (WORD) and offset 0xD for “sectors per cluster” (BYTE).

The cluster size typically corresponds to the size of the disk.  The larger the disk, the larger the cluster size.  My main 250GB drive has a cluster size of 4K.  Originally the drives were small enough to have the sector size and cluster size match (512 bytes).

Back to FSCTL_GET_VOLUME_BITMAP.  When the information is successfully returned from the IOCTL, it reveals the cluster pattern for the volume.  The structure returned is VOLUME_BITMAP_BUFFER which is effectively a bitmap of used/free clusters.  Each byte in this “Buffer” corresponds 8 clusters.  The lowest bit represents the first cluster of that byte.  Just today I figured that if you had 64 bytes of bitmap data, it would correspond to 2MB of data with 4K clusters.  

The actual output of the bytes shows an idea of where the used and free space is concentrated.  As expected, most of the early parts of the disk are used while the last parts are usually free.  There is also hints of fragmentation since there is gaps between sections of data which probably used to be files.

It is actually possible to gather free/used cluster counts from the bitmap by throwing the data through a counter that changes the byte patterns to actual count pairs.  I wrote a program that scanned the whole bitmap using each nibble to match against pre-programmed arrays.  So, put in 0xF and get back 4 used 0 free.  Put in 0×6 and get 2 used and 2 free.  You get the idea.  Originally I had thought of doing it against the byte but was not looking forward to entering the 256 combinations.

I keep on thinking of defrag programs from the past (like Norton) that show the cluster map (from a high level view) and moving files around.  Now it seems fairly simplistic given the amount of clusters involved.  It also seems a bit risky given the temporary nature of the free/used bitmap.

The point there is that the amount of free/used clusters is always changing based on system activity.  A snapshot using the IOCTL is just a picture in time and does not guarantee that things are still the same.  Even Microsoft recommend to assume that you might not get the free clusters you want for a defrag operation so you better be prepared to try again.

The actual information lives inside NTFS in a metadata file called $Bitmap.  It is MFT record number 6 (reserved and for all time the same).  $Bitmap cannot be directly read from any Windows program since it is only intended for the file system.  Obviously Microsoft does not want anyone to change this file.  It would play major havoc on Windows most likely.  

The cluster map in $Bitmap is in theory the basis of what is returned from the IOCTL.  However, based on not being able to do both at exactly the same instant means that they could vary.  The exception to this would be if you could freeze Windows somehow.

Speaking of freezing Windows, the only way to do this successfully is to access the information when nothing is changing.  The easiest way is to access the volume when it is not booted from.  As long as no running program is changing the non-boot drive, it should be possible to get an accurate snapshot that will stay good over time.

Coming from a VHD angle, you could mount the VHD and then use the IOCTL.  Or, you could spend a lot of time understanding the NTFS format along with the VHD format to go get the $Bitmap file yourself.  Difficult, but entirely possible.

Having come to the end of this post, it seems that this topic might be a bit tangential to what most of you might be interested in.  Let’s assume that it is really meant for the tinkerers out there that like to know where the disk space is really being used.  Please expect a few more words about this area in the coming weeks.

Windows XP and June 30 2008

Based on a previous post about Microsoft, it was mentioned in a comment that Microsoft was going to discontinue selling Windows XP on June 30, 2008.  There was news before related to Microsoft postponing the end of sales date for Windows XP by five months.  It does not take long for this to get very confusing.  From one perspective, it looks like it is going to get a lot harder to buy Windows XP after June 30.  However, this view a bit misleading.

First of all, it will be possible for “System Builders” to sell XP on systems until January 31, 2009.  “System Builder” seems a bit unclear but to me it represents people that are willing to build systems by hand and typically would be doing custom work for fairly small sets of customers.  Please feel free to clarify this point.

The main OEMs that license Windows will be expected to switch over to just selling Vista based systems as of June 30.  However, there is a catch (or loophole).  If the business customer deems that they really don’t want Vista, they have the option to downgrade to Windows XP.  They need to buy Vista Business or Ultimate in a volume-license contract before they get this ability.

Microsoft is still planning to support XP until April 2009 (as standard support) and will continue to support security and paid support until 2014 (this seems a bit long actually).

Unlike previous major upgrades (like Windows Me to Windows XP or Windows 3.1 to Windows 95), there has been a lot of back pressure on accepting Windows Vista.  Early adopters have had a variety of experiences but overall the message that came through was that it was best to wait for the service pack for the corporate environment.

Now that the service pack is here,  the feedback is that it resolves many of the early issues and that it is finally ready for more widespread adoption.

However a key point is that most likely you will need a new machine to do this.  Also of note is that there is a good chance that business users will not favour the eye candy when they know that it would work even better on Windows XP.  It is a difficult position for Microsoft to be in.  They want Vista to succeed but the market is not fully convinced.

To this point, I am going to recommend something amazingly simple.

It is common for most hardware corporations to release a product and then go back and think about how they could do it better and cheaper.  This is typically inspired from reducing costs but does have benefits for the customers as well.  Usually the end result performs better and is typically smaller.  Sony has repeated this pattern with the PS2 and PSP.  Both times consumers and producer came out ahead.

In Microsoft’s case, they need to find ways to reduce their hardware cost.  This means that certain assumptions have to be re-examined and the ultimate goal is to produce a version of Vista which is not so hungry and compares much better against XP.  I have heard of nothing like this except for the service pack performing a bit better in certain areas.

There have been examples of this in the past with operating systems as well.  The idea is to make the system more lean and efficient with using resources.  It is very easy to say that programmers/engineers do not always do things the best way the first time around.  Also of importance is that it is hard to see the bigger picture.  If you were to see the Windows code yourself, you would probably judge it as having many sections of duplication and not done with optimal results.

That is the secret of closed source projects.  The warts can hide for years and years and as long as no one raises a stink internally, it will stay that way practically forever.  Open source, however, reveals its flaws almost instantly.  Of course the ugly thing about open source is that it is just as easy to inject new ugliness based on lack of knowledge about the system.

Regardless of this banter, it does seem obvious that Microsoft is competing with XP.  The Vista team needs to realise this and treat it that way.  Instead of silencing the competition, it would be far better to address it directly.