Wednesday, December 21, 2011

STOP: c00002e2 Directory Services could not start


Out of a sudden, one of the Windows 2008 Server Standard x64 Domain Controller encountered the following BSOD error:

-c00002e2 Directory Services could not start because of the following error:

-A device attached to the system is not functioning.

-Error Status: 0x0000001.

Most of the solutions points to the disk where the NTDS is located. In virtualized environments, after changes have been made to the virtual disk setting (for instance a P2V), the second disk (in most cases storing the NTDS) were found offline. The solution would to simply boot into Directory Services Restore Mode, go to the Windows Storage Manager and re-online the affected disk.

However, in my case, I am only using a single disk with a single SYSTEM partition for both my OS & NTDS. Furthermore, in the DSRM, the disk were (surely) online.

Unusually, the solution was to simply backdate the system date in my BIOS (on Virtualized platforms, the virtual BIOS). I backdated the date to a few months back & the Domain Controller booted successfully into the OS.

I can now proceed to restore/replicate/demote/promote my Domain Controller.

Wednesday, August 17, 2011

Password Strength

This comic is saying that the password in the top frames "Tr0ub4dor&3" is easier for password cracking software to guess than "correcthorsebatterystaple". And this is absolutely true that people make passwords hard to remember because that means that they are "safer".

The important thing to take away from this comic is that longer passwords are better because each additional character adds much more time to the breaking of the password.

Steve Gibson from the Security Now podcast did a lot of work in this arena and found that this password 'D0g.....................' is harder to break than this password 'PrXyc.N(n4k77#L!eVdAfp9'. Steve Gibson makes this very clear in his password haystack reference guide and tester:

'Once an exhaustive password search begins, the most important factor is password length!'

That's what xkcd is trying to get through here. Complexity does not matter unless you have length in passwords. Complexity is more difficult for humans to remember. Length is not.

Sunday, April 03, 2011

Microsoft Exchange Server 2010 SP1 on Windows 2008 Server R2 post-installation gotchas: Installing Active Directory Web Services


Active Directory Web Services (ADWS) is an option to enable Web based API access to Active Directory; predominantly via PowerShell and Active Directory Administration Console (ADAC). Read:

This functionality by default is available on Domain Controllers running Windows 2008 R2 and is currently available as add-on for Windows 2003 and Windows 2008 systems (Download from:

NOTE: ADWS uses TCP port 9389; ensure firewalls to and from clients and ADWS-enabled Domain Controllers does not block this port !!!

Microsoft Exchange Server 2010 SP1 on Windows 2008 Server R2 post-installation gotchas

Installation of Microsoft Exchange 2010 SP1 on a Windows 2008 R2 Server should be a straightforward process due to the descriptive Wizard and auto resume of the Setup process. As per the Wizard, following will be the pre-requisites of the Exchange 2010 installation:

  1. Installation of Microsoft .NET Framework 3.5 SP1
    1. This can be installed from Windows Manager – Add Feature – .NET Framework 3.5
    2. .NET would also require basic Web Service (IIS) role as its dependency.
  2. Running ‘setup /PrepareAD’
    1. This would be run automatically by the Setup process
    2. Make sure the account used to run Setup have the proper Domain and Schema Admins permission
  3. Select the language options
    1. Pretty straightforward, just choose use all language available in the DVD or download from the Internet if your language preference is extensive.

After enabling the basic pre-requisites above and resuming the Setup, further verifications by Role will commence. Depending on your existing configuration of the server, this will include:

  1. Enabling IIS Basic, Integrated Windows and Digest Authentication methods
    1. This can be updated from Windows Manager – Roles – Web Service – Add Role Features
  2. Enabling IIS Static and Dynamic Compression
    1. This can be updated from Windows Manager – Roles – Web Service – Add Role Features

After appeasing the requirements above, you should be able to continue the Setup process. Depending whether you are creating a new Exchange organization or adding a new Exchange server in an existing Organization, additional parameters would be required; this would be very straightforward and Setup should complete successfully. Reboot afterwards !!!

That would be the end of the Exchange installation and you should be happily configuring and using your Exchange installation; NOT !!! Depending on the symptoms that may or may not appear, further tweaking would need to be carried out.

NOTE: This may differ between OS versions (2008 SP2 or 2008 R2 SP1) or Exchange versions (RTM, SP1) etc.

  1. You find the Mailbox Database name to be too robotic (Mailbox_Database_1234567XXX…yuck) and decide to create a NEW Mailbox Database only to have error code 0x00000005 “INSUFF_ACCESS_RIGHTS” thrown at you !!! Read here
  2. You try to remove the default/first Mailbox Database and Exchange complains that you need to move out all mailboxes first before the Database can be deleted; problem is, as far as you can see there are no more mailboxes in the database to be moved out !!! Read here
  3. You open Microsoft Exchange PowerShell and in the initial Module loading stage it complains that Active Directory Web Service cannot find any available Domain Controller !!! Read here
  4. You try opening Outlook Web App (OWA) from your Web Browser (e.g: https://exch2010.tld/owa) and nothing comes out; just a blank page !!! Read here
  5. After rebooting the Exchange Server, after logging in to OWA in the login page, you are then presented with an empty page, or worse HTTP error 5XX !!! Read here

Microsoft Exchange Server 2010 SP1 on Windows 2008 Server R2 post-installation gotchas: Microsoft Exchange services start up issues


This is due to Microsoft Exchange Forms Based Authentication service not running; possibly after a normal reboot or power failure.

This is a Microsoft known issue since Exchange 2003 and also affects the Microsoft Exchange System Attendant service. Read and use the solution in

Microsoft reports that this issue affects installations of Exchange on Domain Controllers due to the delayed Global Catalog availability but it is prevalent even on standalone Exchange installations.

The only solution thus far is to configure the affected services with a Automatic Startup (Delayed) as opposed to the default Automatic Startup.

Also enable the 1st, 2nd and 3rd service recovery to attempt startup of the services if startup failed in the first try.

Microsoft Exchange Server 2010 SP1 on Windows 2008 Server R2 post-installation gotchas: Exchange 2010 required Windows Features


Thanks to

After successfully installing Exchange 2010, if you find your OWA page to be blank (but with a successful redirection URI, e.g: https://exch2010.tld/owa/logon/logon.aspx?blablabla) you may not  have all the dependencies installed. Install them using the steps below:

1. Installing dependencies using PowerShell. NOTE: This may reboot the server !!!

   1: Import-Module ServerManager
   2: Add-WindowsFeature NET-Framework,RSAT-ADDS,Web-Server,Web-Basic-Auth,Web-Windows-Auth,Web-Metabase,Web-Net-Ext,Web-Lgcy-Mgmt-Console,WAS-Process-Model,RSAT-Web-Server,Web-ISAPI-Ext,Web-Digest-Auth,Web-Dyn-Compression,NET-HTTP-Activation,RPC-Over-HTTP-Proxy -Restart

2. Installing dependencies using Server Manager:

    1. Open Windows Manager-Features-Add Features

    2. Add the RPC over HTTP Proxy feature; this will auto-install the other required dependencies as the PowerShell method above.

Retry the OWA login via your Web Browser.

NOTE: If the initial OWA login page can be displayed but a blank page is displayed ONLY after submitting the login, this could be due to the Microsoft Exchange Forms Based Authentication service not starting !!! Read here

Microsoft Exchange Server 2010 SP1 on Windows 2008 Server R2 post-installation gotchas: Deleting Arbitration Mailboxes

Exchange mailboxes consists of the normal user/resource Mailboxes and Arbitration mailboxes.

Arbitration mailboxes are “… used for managing approval workflow. For example, an arbitration mailbox is used for handling moderated recipients and distribution group membership approval”. Refer:

To delete a Mailbox database (especially the first created Mailbox database), you need to move out ALL mailboxes first into another Mailbox database.

In a normal Exchange console or from Get-Mailbox command these Arbitration mailboxes would not appear unless supplied with an additional –Arbitration parameter. Thus, in Exchange 2010, to move all mailboxes to another database would require the following command:

1. Moving all normal user/resource mailboxes from DB1 to DB2

   1: Get-Mailbox -Database DB1 | New-MoveRequest -TargetDatabase DB2

2. Moving all Arbitration mailboxes from DB1 to DB2

   1: Get-Mailbox -Database DB1 -Arbitration | New-MoveRequest -TargetDatabase DB2

Wait for the move request to complete and delete/remove the required database.

Microsoft Exchange Server 2010 SP1 on Windows 2008 Server R2 post-installation gotchas: 0x00000005 INSUFF_ACCESS_RIGHTS

This is possibly due to disabled ‘Inheritable permission’ option causing the ‘Exchange Trusted Subsystem’ group not being able to have Full Access to a number of important Microsoft Exchange OUs in the Active Directory configuration dSE.

As Exchange 2010 runs its Active Directory access via the Exchange Trusted Subsystem group permission (not as the logged-on user account permission), relevant objects in the Active Directory would require Full Access rights for this group. This would be (automatically) achievable if the Active Directory objects inherit the permissions from the parent object as the parent’s security permission is changed during Exchange setup’s PrepareAD process.

However, if certain child objects have their Inheritable permission option disabled beforehand, it would not acquire the correct permission level for the Exchange Trusted Subsystem to access them. For resolution, use the steps below:

  1. Using ‘adsiedit,msc’ traverse the Active Directory configuration schema and verify that the following OUs have its inheritable permission enabled (checkout Richard’s Exchange Ramblings blog:
    1. RootDSE-Configuration-Services-Microsoft Exchange-First Organization
    2. RootDSE-Configuration-Services-Microsoft Exchange-First Organization-Administrative Groups
    3. RootDSE-Configuration-Services-Microsoft Exchange-First Organization-Administrative Groups-Exchange Administrative Group (FYDIBOHF23SPDLT)
  2. Remove the Exchange Server computer account from the Exchange Trusted Subsystem group and adding it back again.
  3. Reboot the relevant Exchange server.
  4. Ensure that your currently logged-on account is a member of the Active Directory Schema admins.
    1. In an Administrator elevated Command Prompt re-run Exchange setup’s PrepareAD parameter “%ExchangeInstallationFiles\setup /PrepareAD”
    2. Reboot the Exchange server again.

Wednesday, January 26, 2011

The Linux boot process, a chart - SysAdmin1138 Expounds

The Linux boot process, a chart - SysAdmin1138 Expounds
  • POST
  • Read bootable media
  • Load Master Boot Record
  • Execute MBR
  • POST
  • Read bootable media
  • Load the GPT table
  • Mount the EFI system-partition
  • Run EFI-specific code
  • Stage 1 loaded into MBR/EFI and gets executed by BIOS/EFI
  • Stage 1.5 loaded by Stage 1, including critical drivers
  • Stage 2, in the boot filesystem, executes
  • Stage 2 loads the kernel
  • Stage 1 loaded into MBR/EFI and gets executed by BIOS/EFI
  • Load first sector of core.img
  • Continues loading core.img
  • Loads GRUB config
  • Loads the kernel
  • Stage1 loaded into MBR (or EFI by ELILO) and executed by BIOS/EFI
  • Stage2 is leaded by Stage 1, executes
  • Loads LILO information.
  • Loads the kernel
Kernel Load
  • The kernel uncompresses into memory
  • If configured, the kernel mounts the Initial Ramdisk, which contains needed modules to load the rest of the OS
  • Mounts the root filesystem, loading any needed modules from initrd
  • Swaps / from initrd to the actual root filesystem
  • Executes the specified init process
  • Is launched by the kernel as PID 1
  • Checks /etc/inittab for loading procedures
  • Runs scripts specified by inittab
    • Mounts needed filesystems
    • Loads needed modules
    • Starts needed services based on runlevel
    • Finishes setting up userspace
  • Is launched by the kernel as PID 1
  • Reads /etc/system.conf
  • Mounts needed filesystems
  • Loads needed modules
  • Starts services as needed
  • Is launched by the kernel as PID 1
  • Runs startup events listed in /etc/events.d based on runlevel.
  • Loads needed modules
  • Mounts needed filesystems
  • Starts needed services
  • Is launched by the kernel as PID 1
  • Reads /etc/launchd.conf for config details
  • Reads /etc/launchd.plist for per-driver/service details

Tuesday, January 11, 2011

How I '”usually” bypass transparent content-filters (for troubleshooting purposes)

In my work, transparent content-filtering devices usually throws a (transparent ?) spanner into my troubleshooting work. This usually could be in the form of IPS/IDS devices or transparent proxies. It’s there, happily doing its thing, but quite invisible from end-point device’s perspectives.

My favourite tool in this situation is SSH and its port-forwarding functionality.

Since tunelled traffic is encapsulated and encrypted in SSH transmissions, these pesky transparent device wouldn’t know any better but to allow them through; however, always double-check your Firewall/IPS configurations that it DOES NOT block SSH (default TCP/22) traffic in the first place. Take the following scenario where normal traffic is being (transparently) intercepted and certain policies are applied.


This is what SSH and its port-tunnelling accomplishes:


It pays to have the endpoint devices to be able to ‘talk’ SSH to have the above to work though. The OpenSSH client on *nix systems or Putty on Windows should work perfectly well as the client.

However, native OpenSSH daemon (as the SSH server component) is not easily available on Windows system (in cases where both endpoints are Windows systems). Have a try at freeSSHd, a Windows implementation of SSH server component; it’s easier than trying to run a Cygwin implementation of sshd.

Thursday, January 06, 2011

Updating Sendmail access file (Revisited)

Sendmail access file format for IP addresses is based on full octet wildcards. for example:

  • 10 represent
  • 172.16 represents (sorry, /12 is not available)
  • 192.168.100 represents

A sample of Sendmail’s access file (normally located as /etc/mail/access) is as such:

10                     RELAY
172.16                 RELAY
192.168                RELAY           OK

After editing the Access file, remember to:

1. Run makemap to create/update the access database read by Sendmail from the edited text Access file.

"makemap hash /etc/mail/access < /etc/mail/access"

2. Restart Sendmail

"service sendmail restart" OR "/etc/init.d/sendmail restart"

Wednesday, January 05, 2011

NetBIOS-ssn issue when a naming collision/conflict is detected

I encountered an issue today affecting NetBIOS/CIFS/SMB filesharing, the situation is as follow:
  1. User have one physical Windows 2003 server connecting to a remote CIFS fileserver as mapped drive.
  2. Remote connection is via Firewall and IPS appliances. Firewall only allows TCP/139 outbound, TCP/445 is blocked.
  3. I P2Ved that Windows 2003 server into vSphere
  4. Overall operation of the P2Ved server is OK.
  5. End-user complains that the virtualized server now cannot access the fileshare either directly via UNC or via "net use" command
Unfortunately the end-user did not network-disconnect or powered-off the original physical server, they merely changed its IP address.
This causes the broadcast domain to have a NetBIOS name collision, both the old physical and the new virtualized server uses the same NetBIOS name (but with different IP address).
It seems that when a NetBIOS name collision is detected on the local host, the host refuses to use legacy ports (UDP/137, UDP/138 and TCP/139) and only uses TCP/445 for NetBIOS connection.
Since the connection to the external fileserver has its TCP/445 blocked by the firewall, and since the new server refuses to use legacy NetBIOS ports, i.e.: UDP/137, UDP/138 and TCP/139, the filesharing fails.
Any of the method below should solve the issue:
  • Shutdown or network disconnect the old physical server, Reboot the new server afterwards or
  • Rename the NetBIOS name of the old physical server and reboot the new server or
  • Allow TCP/445 to destination fileserver (destination fileserver must support Microsoft-DS; Windows 2000 and above).

Sunday, January 02, 2011

How I learned to stop worrying and love the game

Once upon a time there was a Principal who thought oh-so highly of one of its Partner.

The Principal gave them price protection that would enable them to win biddings even if other Partners slashed their margin up to 0.1%.

One day, that preferred Partner lost a bidding to an oh-not so preferred Partner. End of story? Not really.

The not-so preferred Partner requested for a consolation discount from the Principal; anyways, their Product still got sold and the Principal still retained good margin on the Product. Better margin than if the Product was sold along with the protected pricing of the preferred Partner.

Having good relationship with the end-users, the not-so preferred Partner then proceeded to pass the contact details of the end-user to the Principal to plead their case.

The Principal made a big hoo-haa of the oh-not-so preferred Partner NOT doing any groundwork in the project and thus not entitled to any discounts and rather than contacting the end-user, they proceeded to contact the preferred Partner (who lost the bid) and ask regarding the outcome of the bidding.

The clueless preferred Partner informed the more clueless Principal that the project is not yet awarded and the winner of the bid is still not decided by the end-user.

Quite odd as the oh-not-so preferred Partner even attached a copy of the Award letter when requesting for the consolation discount.

Sometimes, IMHO, even FreeBSD jail looks more attractive technically and Xen looks more attractive financially than this virtualization Product carried by this clueless Principal.

And they lived happily ever after..... the end.