“SafeBoot is corrupted (92h)” when McAfee Endpoint Encryption is installed

Here’s a rare example of a situation where I was actually very close to contemplating a reformat before finally stumbling across an idea which led to its resolution.

The client was an employee of a large company whose laptop hard drives (like many) are encrypted using McAfee Safeboot (Endpoint Encryption).  This is a great strategy for protecting against data theft in the event that the machine is stolen, but like all data encryption, any sort of problem whatsoever with regard to either data integrity or the encryption/decryption process can be potentially disastrous.

Since SafeBoot/EE is a boot sector encryption (“pre-boot authentication” it’s called) which applies to the entire storage volume, any problems relating to its boot partition information can completely break the entire system until either the information is repaired or the volume is manually decrypted.  Fortunately, McAfee provides a method for decrypting volumes offline using a special boot CD (called SafeTech/WinTech, depending on your version) and a rolling Authorization Code which changes daily.  Unfortunately, they only make these tools available directly to clients… meaning that us independent techs are completely out of luck, even if we have the decryption key (which the customer provided me).

While I was able to bypass the rolling Authentication Code by adjusting the system’s date to a specific day years back and using a code I found posted on an online forum, the built-in tools and even a SafeBoot Admin CD I located weren’t helpful in resolving the issue.  We had already tried contacting the IT department, who says they had no such tools at their disposal and that a reformat would be necessary.  Worse yet, my client was only in Louisville on a brief business trip, and there was data they needed on the encrypted volume.

The next morning, I had planned to contact the customer and suggest that we do just that: reformat.  But on a last-minute whim, I stumbled across an idea which could perhaps explain the strange behavior we were experiencing.

If you Google the Safeboot 92h error, you’ll find a variety of reasons why it can occur, and plenty of situations where it was never properly resolved.  Reverse-engineering the situation in my mind, however, led me to an important point which I had almost disregarded in the midst of my desire to quickly resolve the situation and return a working PC to my busy traveling executive: the possibility that malware had triggered the problem to begin with.

You see, the very first step I performed was a drive image of the encrypted data for safety.  And I had noticed that, throughout the course of the imaging process, no read errors/bad sectors had been encountered.  For surety, I even performed a hardware diagnostic subsequently, and no memory or hard drive errors were found.  This means that the encryption probably had not failed due to a hardware issue, and that instead, some sort of software was likely to blame.  Malware is the perfect culprit here; most notably, boot sector rootkits, which specifically alter the MBR and/or partition information of a victim’s machine to provide a filtering mechanism for the malware where it is also entirely undetectable.

One such rootkit you’ve already seen me write about in the past: Rootkit.Boot.Pihar.B.  It creates a hidden, encrypted partition at the end of a drive and sets it as the main Active partition, after which it chainloads the operating system partition (which is set as Inactive by the rootkit).  This effectively allows the rootkit to leave the MBR “clean” by storing all code in its encrypted partition and then loading the OS normally afterwards.  While this rootkit wasn’t currently detected as active on my client’s system (using an offline TDSSKiller run), after booting to a partition management software, I found that the hidden/encrypted partition was present.  And the OS partition, predictably, was Inactive.

It was just that simple.  Setting the OS partition as Active and then booting normally did the trick.  Following that, I had plenty of malware cleanup on my hands, most of it related to a rogue which had piggybacked on the rootkit.  This was a lot of fun since the machine’s Administrator restrictions and policies stood in the way, but in the end, the machine was repaired, all data was recovered, and my client was happy.  That is, until he returns to his hometown, where he’ll then request an updated Windows 7 machine to replace this dinosaur. 🙂

Solution: STOP Error 0x0000007b (0xfffff880009a98e8 0xffffffffc000000d 0x0000000000000000…)

Hoo boy, this one’s a doozy.

So following the removal of certain rootkits (such as Rootkit.Boot.SST.a, which is associated with the Windows Recovery rogue), you may find that your Windows boot configuration data has been totally corrupted. Worse yet, the usual steps to remedy (such as those described in my earlier post about TDL4 and the resulting blue screen) all fall apart when you reach the bootrec /RebuildBCD command, which returns the message:

Total identified Windows installations: 0

Geez.  This essentially means that the bootrec command cannot identify your Windows installation, even though the Windows Recovery Environment has no trouble doing so upon starting.  So, now what?

Sometimes it’s as simple as opening up your favorite disk partitioning software and marking the C: partition as ACTIVE, and if there are still problems, subsequently recovering the boot data as I mentioned in the TDL4 post (keep in mind however that the System Managed partition is typically Active normally on a Windows 7 system thanks to the isolated boot partition that it uses).  This problem occurs because of some modern rootkits which create a hidden, encrypted partition at the end of the system drive and mark it as Active and Primary (while simultaneously marking the standard boot partition as Inactive).  This infection has been covered in other recent blog posts as well.

Sometimes, however, the BCD is totally corrupted and this doesn’t even work.  At this point, most every source on the internet comes up a dead end.  Everyone ends up reformatting or reinstalling Windows overtop their existing partition; nothing else seems to work.

You might not think it’d be helpful, but there’s an intimidating post over at the EasyBCD NeoSmart site which explains how to manually rebuild the Vista bootloader from the ground up in catastrophic situations.  As it turns out, this procedure applies to Windows 7 as well (which uses the same bootloader and BCD structure), and it’s the key to your recovery here.

It’s no easy feat however, so roll up your sleeves and get ready to do some typing.  Here’s the full procedure from start to finish:

  1. Boot to the Windows Recovery Environment either by selecting Repair Your Computer when Windows fails to boot, by inserting the Windows installation disc, or by using a Windows ERD/MS DART disc (if you happen to have access to one, that is).
  2. Cancel the recovery attempt if it tries to start on its own (it will fail anyway) and then choose the advanced options link at the bottom of the window.
  3. Choose to open the Command Prompt.
  4. Here’s the fun part.  Once at the prompt, enter the following commands one by one.  Take care not to mistype anything, and be sure to replace C: with whatever your system drive happens to be:

bootrec.exe /fixmbr
bootsect.exe /nt60 all /force
bcdedit /export C:\BCD_Backup
attrib -h -s C:\boot\BCD
ren C:\boot\BCD BCD.old
bcdedit /createstore c:\boot\bcd.temp
bcdedit.exe /store c:\boot\bcd.temp /create {bootmgr} /d “Windows Boot Manager”
bcdedit.exe /import c:\boot\bcd.temp
bcdedit.exe /set {bootmgr} device partition=C:
bcdedit.exe /timeout 10
attrib -h -s C:\boot\bcd.temp
del c:\boot\bcd.temp
bcdedit.exe /create /d “Windows 7” /application osloader

At this point, note the value within the curly braces {……..} as you will need it during the next steps.  Replace the dots within the curly braces below with that entire string on each line.  NOTE:  To make this easier, once you type it once, you can press the Up arrow to restore the last command and simply edit that line for the next one.

bcdedit.exe /set {…..} device partition=C:
bcdedit.exe /set {…..} osdevice partition=C:
bcdedit.exe /set {…..} path \Windows\system32\winload.exe
bcdedit.exe /set {…..} systemroot \Windows
bcdedit.exe /displayorder {…..}
bcdedit.exe /default {…..}
bcdedit.exe /set {…..} locale en-US

Thanks to Bitt Faulk for the final line, which restores the correct Windows loading screen as well.  You will need to replace the en-US entry with something different representing your region if you are not in the US.

Then you’re back in Windows, miraculously.  No reinstall necessary!

Side effects?  A little.  Hopefully you can handle not having the nifty new Windows 7 startup animation screen, because this will lose it for you.  Instead, you’ll be stuck with the old-school plain Jane Windows Vista progress bar.  You’ll also lose any special boot options you had previously.  But as a last resort, this works, and it’s still just as quick as ever.

Apart from that, once you’re back in Windows, of course, you’ll still have to disinfect the rest of the way.  In my customer’s case, the system damage was actually so bad that I ended up performing an in-place upgrade (the Vista/7 equivalent of a Repair Install), but after that, everything was great.  It was a triumph for sure, and yet another situation where the usual solution of reformat/reinstall was not necessary.  Now you know how to avoid it!

I hope you’ve found this post useful–if so, please take a moment to leave me a comment!

If you need computer help in the Louisville, KY area, there’s simply no one better.  Give me a call today!

STOP: c0000135 – The program can’t start because consrv is missing. Try resintalling the program.

Getting this error?  Wouldn’t you know, it’s actually the product of a nasty little rootkit called ZeroAccess MAX++ of which you might be familiar.  The particular variant that causes this error actually uses the consrv.dll file to ensure it is able to load at boot on 64-bit systems.  As such, among other items, this rootkit drops a file called consrv.dll into the %SYSTEMROOT%\system32 folder.  It’s the reference to this file in the registry which wreaks said havoc once the file is removed by any means (antivirus, offline deletion, etc.).

To rectify the problem, you will need to gain access to the %SYSTEMROOT%\system32\config\SYSTEM registry hive remotely (whether by Recovery Console’s regedit and the Load Hive… command or by booting to another operating system and loading the hive in similar manner) and change an entry modified by the rootkit back to its default value.

The infected machine will have a modified string (REG_EXPAND_SZ) data of the Windows registry value in HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Session Manager\SubSystems which looks like this:

%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=consrv:ConServerDllInitialization,2 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16
This value is wrong, and it’s the reference to consrv which is generating your c0000135 stop error.  Instead, change it to:
%SystemRoot%\system32\csrss.exe ObjectDirectory=\Windows SharedSection=1024,20480,768 Windows=On SubSystemType=Windows ServerDll=basesrv,1 ServerDll=winsrv:UserServerDllInitialization,3 ServerDll=winsrv:ConServerDllInitialization,2 ServerDll=sxssrv,4 ProfileControl=Off MaxRequestThreads=16

This will solve the problem and enable the machine to boot.  Please note that you should also modify the same value in HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\… for completeness.

For a little more background/ancillary info, on the machine I was repairing which experienced this issue, the rootkit was accompanied by an additional MBR bootkit of the family Rootkit.boot.Pihar.a.  TDSSKiller actually took care of this one and restored the clean MBR.  In addition, plenty of other malware was along for the ride (all of it predictably hidden by the rootkit combo).  It was a nasty situation, but nothing I couldn’t handle. 😉
I hope you have found this case study useful.  Please let me know if it has helped you!

TDL4 removal leads to Windows 7 64-bit stop error on boot

I’m proud to say that it’s been literally two years since I’ve reformatted a PC due to malware, and I’ve disinfected many hundreds in that time period. But yesterday, I encountered a PC infected with TDL4 (which I’ve dealt with many times), however this one was a Windows 7 64-bit machine. Following removal of the rootkit offline via TDSSKiller in WinPE (set to scan Boot Sectors only), the PC began crashing on every boot, even when Safe Mode was attempted. The typical invasive offline procedures I use to rectify these issues — such as the disabling of nearly all third-party filesystem and NDIS filter drivers — did nothing to correct the issue. Restoring a previous System Restore point and even manually restoring the registry hives from the RegBack folder also accomplished nothing.

Finally, I decided to backup/restore the BCD. To do this, I used a Windows 7 recovery disc (in my case, ERD/MS DART) and opened a command prompt. From there, I (mostly) just followed the Microsoft guidelines for restoring a corrupted BCD:

bcdedit /export C:\BCD_Backup
attrib -s -h c:\boot\bcd
ren c:\boot\bcd bcd.old
bootrec /fixmbr
C:\boot\bootsect.exe /nt60 all /force
bootrec /RebuildBcd
bootrec /FixBoot

(Note that C: must be replaced with the relevant Windows drive.)

This corrected the issue on this PC. I hope this helps someone in the future!

Post-disinfection Stop Error: c000021a

On multiple occasions over the past couple of years (since kernel-patching rootkits have become prevalent), I have encountered a particularly troubling STOP Error that most often technicians simply respond to by running a repair or clean install of the operating system. It looks something like this:

STOP: c000021a {Fatal System Error}

The windows Logon Process system process terminated unexpectedly with a status of 0x[various] (0x[various] 0x[various]). The system has been shut down.

Where [various] represents any number of hexadecimal error codes that may apply.

In this case, simply restoring the registry to a previous state as I’ve written about before does not correct the problem.

There are posts and pages to address this situation scattered across the internet, but nearly all of them offer different solutions, and most of them don’t seem to work. Microsoft has a page that relates the issue to Windows NT OS, blaming it on the PendingFileRenameOperations registry key which is often used by malware and antimalware to perform a rename operation on reboot. However, fixing this key as they suggest also does not solve the problem.

Most often, when it’s encountered post-disinfection, I find that the problem relates to an issue with a patched system file. I’m not certain, but it nearly always seems to be winlogon.exe.

Regardless, there is one and one only way to ensure the problem is corrected: find and replace the suspect file! There are multiple ways to accomplish this:

  1. Boot to a custom OS or slave the drive, check the system files (or run a virus scanner through them), and replace the faulty files with good copies (see below).
  2. Run a system file check from the Windows Recovery Console by typing sfc /scannow.
  3. Boot to an MS DART ERD Commander disc for Windows XP and run a system file repair from within the environment (this is my method of choice).

If you are forced to manually inspect and/or replace the files, I suggest checking for Company information and investigating any files which are missing the usual Microsoft Corporation info. If a suspect file is encountered, check its MD5 hash and Google it to see if it’s a known patched copy.

Once the culprit(s) has been identified, navigate to the system32\dllcache folder and copy the corresponding file there to its correct location. If it isn’t there or the copy is also bad, restore it from the Windows CD, or look for a folder called \i386. You can also run a search for the file throughout the entire Windows directory to find copies which have been downloaded for Service Packs and other Windows Updates. Just be sure to get the correct version.

In my most recent customer’s case, the files at fault were explorer.exe and winlogon.exe. Both had been patched by a rootkit and needed to be replaced. Once that was finished, the system booted up just fine.

If you’re looking for computer help in the Louisville area, look no further.  Call me today and get it done right!