Archive for the ‘Windows Server 2008 R2’ Category

Robocopy can silently fail to copy directories with invalid UTF-16 names (or, why I always compare after copying)

December 11, 2014

Over the last few days I’ve been copying the bulk of the home directories on my primary file server over to a new volume (don’t ask) and, of course, I did a comparison afterwards to make sure the copy was successful [1].  I’m talking about 3070 home directories, comprising over seven million files structured in any number of strange and wonderful ways.

I wasn’t at all surprised to find that 3069 of those directories had copied perfectly; robocopy is pretty reliable.

I was a little surprised to find that one directory had an anomaly, but still, glitches happen.  I became puzzled, though, when I realized what the problem was: an entire subdirectory was missing.  Robocopy hadn’t reported any errors.  What’s more, when I ran robocopy over that home directory again, it reported that there was nothing to do: as far as it was concerned, source and destination were a perfect match.

Explorer didn’t show me much.  The name of the two directories in the source looked the same; the first character was shown as a box.  Another little tool of mine, though, could see the difference:


The tool escapes non-ASCII characters with a percent sign followed by a hexadecimal representation, so the first wide character is 0xD898 in the first directory and 0xDADB in the second.  Otherwise the names are the same.  Only the first one was present in the destination.

The next step, obviously, was to look up the Unicode code points 0xD898 and 0xDADB.  As it turns out, they are “high-surrogate code points”, used in UTF-16 to encode Unicode code points larger than 16 bits.  The key here is that surrogate code points are only valid in pairs: an individual surrogate code point is meaningless.

Of course, NTFS doesn’t care.  It doesn’t really understand Unicode, so one 16-bit character is much like another.  As far as NTFS is concerned, those are perfectly good (and distinct) names.  Robocopy, however, must for some reason be converting or normalizing the UTF-16 strings, and as a result it sees those two names as identical.  (It appears to be ignoring the second occurrence of the “same” name in a single directory; it doesn’t attempt to copy the second subdirectory at all.)

So, if you’re in the habit of creating files with invalid UTF-16 names, be warned. 🙂

[1] Using some code I wrote myself.  Microsoft don’t seem to have provided a reliable directory-level comparison tool, and I’m not aware of any existing third-party solutions.  I should open-source that tool one day.


Installing 32-bit software as SYSTEM in Windows 7 x64

February 20, 2011


I recently ran into an issue updating the Sun Java runtime on our x64 machines.  We don’t have the budget for fancy deployment solutions, so we just use a startup script (actually an executable, but that’s just fine-tuning) that checks the version number and runs the installer(s) as necessary.

Installing the 32-bit JRE results in error code 1619, which NET HELPMSG translates as “This installation package could not be opened. Verify that the package exists and that you can access it, or contact the application vendor to verify that this is a valid Windows Installer package.”  Running the installer in interactive mode produces the same message.  The installer works normally when run from the context of a logged-in user.

Several hours of troubleshooting later, I identified the source of the problem.  Startup scripts run as local system.  In Windows 7, processes that run as local system have a special profile found in c:\windows\system32\config\systemprofile.  Unfortunately, on 64-bit systems, there are two system32 folders; one for 64-bit processes,and another (whose real name is syswow64) for 32-bit processes.  As a result, there are two separate system profiles; one for 32-bit, one for 64-bit.

So what?  Well, the Sun Java installer unpacks into a subfolder of the LocalLow application data directory.  In this case, the folder in question is c:\windows\system32\config\systemprofile\AppData\LocalLow\Sun\Java\jre1.6.0_24.  Because this is a 32-bit process, though, it is really writing to syswow64 instead of system32.

The Windows Installer, however, is a 64-bit process.  So when it is asked to open the MSI file, it’s looking in the wrong place; hence error code 1619.  The file can’t be opened because it can’t be found.

The same underlying problem (duplication of the system profile) seems to be the cause of this problem with Known Folders warning 1002 appearing repeatedly in the event log.  Some 32-bit system process is registering folder paths inside the (32-bit) system profile and of course these folders can’t be found by 64-bit processes.

For the problem with installing 32-bit software, there are a number of possible workarounds.  You could manually extract the installer files, copy them to a suitable path on the local system, and run them directly.  Most installers won’t mind this, although some will balk or fail to function properly.  Alternatively (and this is the solution I chose) you could create the necessary directory ahead of time and add a junction point (mklink /J) from the 64-bit profile to the 32-bit profile (note that this command line assumes you are in a 32-bit context, and has been split for readability):

mklink /J c:\windows\sysnative\config\systemprofile\AppData\LocalLow\Sun\Java\jre1.6.0_24

This is the equivalent command if you are in a 64-bit context:

mklink /J c:\windows\system32\config\systemprofile\AppData\LocalLow\Sun\Java\jre1.6.0_24

Another possible approach would be to merge the two system profiles together and create a junction point from one to the other.  That would solve this issue for all installers, as well as the Known Folders issue and any other variants.  However, I can’t recommend doing this; it’s too broad a change, and there’s no way to predict what it might break.  If you’re very brave, go ahead, but test thoroughly – and don’t blame me!

Hope this helps.


DISM corrupts images mounted to paths with short filenames

February 22, 2010

If you’ve been having trouble using DISM to update off-line Windows 7 images, make sure you aren’t mounting the image to a path which contains a short filename component.  In particular, note that the TEMP environment variable will contain a short filename component in Windows 2003 or earlier, and may contain a short filename component in later versions, depending on your username.

For example, if I were to create a folder on C drive named harryjohnston, this command would corrupt the mounted image:

dism /mount-wim /wimfile:c:\win7\install.wim /index:1 /mountdir:c:\harryj~1\mount

This wouldn’t:

dism /mount-wim /wimfile:c:\win7\install.wim /index:1 /mountdir:c:\harryjohnston\mount

If an image has been corrupted in this way, and is then used to install Windows 7 on a machine, the junction points on the target machine will retain the path of the mountpoint.  For example, the “Documents and Settings” junction point would point to c:\harryj~1\mount\Users instead of to c:\Users as it should.

Thanks to Joseph Conway from Microsoft, this issue has been filed as a bug and will hopefully be fixed in a future release of the WAIK (Windows Automated Installation Kit).  In the meantime, the workaround is obvious: use a different mountpoint. 🙂

Hope this helps.

The Provision a Shared Folder Wizard Changes Quota Settings

December 28, 2009

In Windows Server 2008 R2, if the Provision a Shared Folder Wizard is used to share a folder which has a folder quota defined, the quota’s source template is reapplied.  If the quota did not match its  source template, the quota settings will be changed unexpectedly.  If the quota was not created from a source template, the first available template will be applied.

You can select a particular template to apply instead of the source template/first available template, but you cannot prevent a template from being applied.

To work around this problem, create a template that matches the quota on the folder being shared before running the Provision a Shared Folder Wizard, and select this template at the appropriate point.