Category Archives: Windows

LAPS presentation

Last December I was presenting at Moscow IT Pro UG about Microsoft LAPS. Finally, the record of the presentation is processed and available at YouTube:

Do not forget to enable English subtitles (they are NOT auto-translated :D)

You may find PPTX-file here.

After the presentation, I was asked a couple of questions:
Q: Does the GUI tool send the password in plain text over the network?
A: No, LDAP connection is encrypted with SASL. But if you are going to access the password attribute in your own scripts/tools, you may accidentally expose the password if you set LDAP_OPT_ENCRYPT = 0 or will use ldap_simple_bind w/o TLS/SSL.

Q: Why do we even need those local accounts? Why not to disable them completely?
A: In case a machine has lost its connection to AD (due to network configuration change, for example), you may want to bring it back on line w/o disruptive actions such as reboot, offline password reset etc. In case if you hope that the machine keeps your offline password hash, you may find yourself in a stressful situation, when you find out that it, in fact, does not.

SCDPM in a tiered infrastructure

When a company has a secure infrastructure, usually there are several tiers of resources managed by different administrators (or, at least, by same administrators but using different user accounts). For example, one may separate sensitive servers, like PKI Certification Authorities, Hyper-V hosts or file servers containing PII, and mark them as Tier 1 servers, while marking all other servers as Tier 2. Then he sets up permissions in a way that each tier has its own local administrators, and you may even forbid cross-tier logon completely (except network logon – network logon is useful and doesn’t pose a security threat).

In the ideal world you would have separate management solutions for each tier. But we all live in real world and, sometimes, it is impossible to find additional resources to support your infrastructure. In that case, it is more appropriate to designate your management servers, including backup ones, as Tier 1 – this way more secure servers will be able to access resources residing on less secure servers but not vice versa.

What does this mean for SCDPM? DPM wasn’t designed to backup resources from another security tier, but we can bent it to our will.
After you install an SCDPM agent on a server in Tier 2, then you must attach it to an SCDPM server in Tier 1. At this step, a user, which you are using to attach the agent, must be a local administrator at both the server and the client. Considering our tiered infrastructure, this is impossible, as one user cannot be a member of local administrators on machines from different tiers.
Fear not! We shall grant required permissions granularly in two steps:

Step 1

Basically we need to allow following permissions for Tier 1 admin at Tier 2 server’s WMI root and propagate them through the tree:

  • Enable
  • MethodExecute
  • RemoteAccess
  • ReadSecurity

You may choose to assign these permissions either via GUI, using wmimgmt.msc, or using PowerShell.
For PowerShell way you may use this fixed version of Set-WmiNamespaceSecurity.ps1 script. Original, written by Steeve Lee, suffers from a bug which does not allow to set inheritance flag and throws an error: “Invoke-WmiMethod : Invalid parameter”.
Run PowerShell script at the Tier 2 client as follows:
Set-WmiNamespaceSecurity.ps1 -namespace 'root' -operation 'add' -account 'EXAMPLE\tier1-admin' -permissions 'Enable','MethodExecute','RemoteAccess','ReadSecurity' -allowInherit $true

If you are going to set permissions with a GUI, here’s how it should looks like:

Step 2

This is counter-intuitive one: As we know, SCDPM server requests the time zone from an agent and saves it in the database. Sometimes, somehow, step 1 is not enough for remote non-admin user to request computer’s time zone. As a workaround, execute following WMI query at an SCDPM client: select * from Win32_TimeZone. After that, remote non-admin user will be able to request TimeZone instances for some time.
To utilize PowerShell for the task, execute this: Get-WmiObject -Query 'select * from Win32_TimeZone'

After these two steps, you should be able to add Tier 2 agent under protection of Tier 1 SCDPM server. When you have finished, you may safely remove those permissions by running the following command:
Set-WmiNamespaceSecurity.ps1 -namespace root -operation delete -account 'EXAMPLE\tier1-admin'

Split-brain DNS Synchronizer

The latest version of the script available at GitHub.

Many companies use the same domain name for both internal and external servers hosting. When an internal domain name is a name of AD DS domain, and internal users must access some of the servers by their external IP-addresses, the problem arises: somehow all these external names must exist in the internal zone and the record information in these internal records must be in correspondence with the external ones. There are several possible solutions to resolve such situation:

1. Blindly forward all external requests to AD DS controllers. In this case, domain controllers are the primary name servers for the zone.

Pros:

  • Single point of management.
  • No new software on domain controllers required.

Cons:

  • You cannot point internal and external users to different hosts for the same DNS-record.
  • Requires to allow external users access to internal servers, which might be impossible due to security policies. Possible exposure of internal infrastructure to an external malicious user.
2. Have two separated set of DNS-servers for internal and external zones.

Pros:

  • You can point internal and external users to different hosts for the same DNS-record.
  • External users do not access internal servers.
  • No new software on domain controllers required.

Cons:

  • Two different points of management. DNS-records may become outdated.
3. Use DNS policies – the new Windows Server 2016 functionality.

https://blogs.technet.microsoft.com/teamdhcp/2015/08/31/split-brain-dns-in-active-directory-environment-using-dns-policies/

Pros:

  • Single point of management.
  • You can point internal and external users to different hosts for the same DNS-record.

Cons:

  • Requires domain controllers migration to the latest software, which is not possible for some organizations.
  • Requires to allow external users access to internal servers, which might be impossible due to security policies. Possible exposure of internal infrastructure to an external malicious user.

 

Currently, I prefer the second method, with two sets of DNS servers. But in that case we have another challenge: How to ensure that all DNS-records, which must point to the same location for both external and internal users, are in sync?
Continue reading Split-brain DNS Synchronizer

How to allow users to join their computers into AD domain

Imagine, that half of your company users are local administrators at their machines. They pretty often reinstall operating systems and request IT Service Desk to join that newly installed OSes into corporate Active Directory domain. One usually has two options to help the users: either to come to the user’s workstation and use one’s credentials to join a PC into domain, or to recreate workstation’s computer account, at the same time allowing employee’s user account to join the computer into a domain by him-/herself. In the first case, ServiceDesk employee needs to walk to a user’s desk, which may be quite exhausting, especially for remote locations, in the second case, computer’s group membership will be probably lost and a new account must be added into all appropriate groups manually.
Is it possible to decrease time and effort put into resolution of such requests? Yes, absolutely!

The main trick is to assign a user with following permissions to his computer’s account:

  • Validated write to DNS host name
  • Validated write to service principal name
  • List the children of an object
  • Read
  • Read security information
  • List the object access
  • Control access right
  • Delete an object and all of its children
  • Delete
  • Write to the following properties:
    • sAMAccountName
    • displayName
    • description
    • Logon Information
    • Account Restrictions

User with abovementioned permissions will be able to join their PC into an AD domain without any assistance from Service Desk.

I made this little script to automate the permissions assigning. Please look into the help section (or use Get-Help cmdlet) to find out about its syntax and usage examples.

In case you use Windows 8.1/Server 2012 R2, you might need to install KB 3092002, either way, only member of the “Domain Admins” group will be able to execute the script. This is due to a bug in the Set-Acl cmdlet. The fix for Windows 10 is included in the latest RSAT package.

If you unsure how to use the script or experience any errors, please leave a comment below or contact me directly.

“setspn -x” is case-insensitive now

As you probably know, duplicate SPNs cause Kerberos authentication errors in AD DS domains. You may notice it by looking for KRB_AP_ERR_MODIFIED errors and Event ID 11 in system logs. With Windows Server 2008, Microsoft released a largely improved version of setspn, which includes “-x” switch to help you proactively monitor your infrastructure for duplicate SPNs. Combined with “-f” switch, setspn output contains duplicate SPNs not only from a single domain, but from a whole AD DS forest. Many companies rely on a result of “setspn -x -f” command as a data source for monitoring systems.

Today I found, that all these years almost nobody noticed that “setspn -x” command compares SPNs case-sensitively, i.e. following SPNs will be considered different and will not be shown in the output:

  • HOST/SERVERNAME
  • HOST/ServerName

Starting from Windows 10, Microsoft changed behavior of setspn to case-insensitive, and, from now on, every duplicated SPN will be displayed in setspn output, disregarding its case.

While Microsoft asserts, that Windows is case-insensitive to SPNs, not every Microsoft product agrees: for example, Shane Young found that you must pay attention to SPNs used by SharePoint accounts.

As a conclusion, I suggest every AD DS administrator to check their infrastructure with the setspn tool shipped with Windows 10, at least once. It allows you to find TRULY EVERY duplicate SPN (I did found a couple, myself ;).

AuthenticationSilo claim is not issued

You setup an Active Directory Authentication Policy and use a membership in Authentication Policy Silo as an access control condition. Next you setup Authentication Policy Silo to use the abovementioned Authentication Policy for appropriate principal types. You set the silo into “audit-only” mode.

In that case, AuthenticationSilo claim is not issued for your security principals.

Why does this happen?

As described in 3.1.1.11.2.18 GetAuthSiloClaim section of Active Directory Technical Specification, AuthenticationSilo claim is issued only when policies in Authentication Silo are enforced:
/*
Check if user is assigned to an enforced silo.
*/
assignedSilo := pADPrincipal!msDS-AssignedAuthNPolicySilo
if (assignedSilo = NULL ||
assignedSilo!msDS-AuthNPolicySiloEnforced = FALSE)
return NULL
endif

Resolution

I’ve found no option to modify this behavior yet. Just keep it in mind while you are testing your Authentication Policies configuration.

«0x80070721 A security package specific error occurred» while using WMI between domains

Symptoms:

You have two IBM System x servers with Windows Server 2012 or newer and with «IBM USB Remote NDIS Network Device» network interface enabled. Both of these servers reside in the same AD DS forest but in different domains. You try to setup a WMI session from one to another (using wbemtest, for example). In that case WMI connection fails and you receive an error:

Number: 0x80070721 Facility: Win32 Description: A security package specific error occurred.»
In System log of the server which initiates the connection we have:
Log Name: System
Source: Microsoft-Windows-Security-Kerberos
Date: 10/6/2013 9:13:09 PM
Event ID: 4
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: SRV1.alpha.example.com
Description:
The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server srv1$. The target name used was host/srv2.beta.example.com. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (ALPHA.EXAMPLE.COM) is different from the client domain (BETA.EXAMPLE.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.

Why does this happen?

Starting from Windows Server 2012, when one machine connects to another computer’s WMI, Windows asks remote system for IP-addresses of its network interfaces. Then operating system chooses which one of them serves its needs to connect best.
In case of IBM System x, both servers have network interface with the same IP-address – 169.254.95.120. Windows chooses this IP-address as the best and tries to connect to it. But instead of remote system, it connects to itself and you see the error.

When you try to connect with wbemtest, it calls a WMI API. In the background WMI uses DCOM to communicate with other servers. When DCOM establishes a session it uses «ServerAlive2» query to check the other server.
Here is what network capture looks like:

14 14:38:55 10.01.2014 0.0111483 srv2.beta.example.com 135 (0x87) 91.103.70.14 55535 (0xD8EF) DCOM DCOM:IObjectExporter:ServerAlive2 Response {MSRPC:10, TCP:9, IPv4:8}
NetworkAddress: srv1
NetworkAddress: 169.254.95.120
NetworkAddress: 192.0.2.1

Same traffic when we access from another end:

38 14:37:12 10.01.2014 37.1871388 srv1.alpha.example.com 135 (0x87) 10.253.12.2 59278 (0xE78E) DCOM DCOM:IObjectExporter:ServerAlive2 Response {MSRPC:14, TCP:13, IPv4:8}
NetworkAddress: srv2
NetworkAddress: 169.254.95.120
NetworkAddress: 203.0.113.2

The application layer chooses the common IP-address and loops back the real connection. The DCOM gets a Kerberos ticket, and tries to authenticate with it, so this is why we see AP_ERR_MODIFIED errors from Kerberos. So a DCOM based communication (for example WMI) won’t work if the participants has a common IP address.

This problem is known by Microsoft and will not be fixed since it is by design.

How to mitigate it?

  1. Just disable IMM USB Network Interface at one or at both servers. But beware: updating of IMM firmware from Windows or using ASU64 will enable this interface again. If you choose this option, I suggest you to setup monitoring to alert you when that interface leave enabled.
  2. Change IP-address at one if the interfaces to something else. You can use almost any private address but here are recommendations from IBM. You even can use DHCP-server to achieve it (I hope you are using separate VLAN for management interfaces, aren’t you?).

Process Monitor 3.10

An important update of Process Monitor was released couple of a days ago:

As you may know, we can use two functions with completely opposite names to open registry key: RegCreateKeyEx and RegOpenKeyEx. When you use RegCreateKeyEx, it creates registry key if it’s non-existed, but just opens it if key exists. RegCreateKeyEx writes which operation (create or open) it’s performed into a separate variable.
RegOpenKeyEx cannot create registry keys and returns error if key doesn’t exist.

Before this release there were no way to determine what operation exactly RegCreateKeyEx perform. “Granted Access” property for execution of that function always contained “Read/Write” value.

From this last update, Process Monitor finally can show you what RegCreateKeyEx does. There is no “Granted Access” property for RegCreateKey operation anymore, it was replaced with new “Disposition” property. “Disposition” may contains following strings:
REG_CREATE_NEW_KEY – if new registry key was created.

REG_OPENED_EXISTING_KEY – if RegCreateKeyEx just opened previously existed key.

“Desired Access” property still contains “Read/Write” value, because we cannot predict which action RegCreateKeyEx will do.

Using this new feature, you can separate RegCreateKeyEx calls: just add new condition into Process Monitor’s filter with following parameters:
Column – “Detail”
Relation – “contains”
Value – “REG_CREATE_NEW_KEY” or “REG_OPENED_EXISTING_KEY”
Action – “Include”

Unable to find the callback library jcb.dll (or one of its dependencies)

If you need to defragment your Exchange database at a computer without Exchange components, you will probably use this MS KB: 244525: How to run Eseutil on a computer without Exchange Server. I tried it when I was defragmenting Exchange 2003 database at a Windows Server 2008 R2 machine.

Eseutil was running fine, but after some time (when it used just over 2 GBs of RAM), I received an error described here: 273087: Error With Jcb.dll While Running Eseutil. Unfortunately, none of the methods described there were useful: when I pressed «Cancel» button, I received an «Operation terminated with error -2102 JET_errCallbackNotResolved, A callback function could not be found) after 1168.136 seconds.» error.

Eventually, I grabbed Process Monitor and found out that files described in Microsoft’s KB 244525 and in this thread «Re: JCB.DLL Not Found Error» weren’t enough – you need another one file: ntlsapi.dll. I copied it from a nearest Windows Server 2003 R2 SP2 box to a system where eseutil were working and everything went smoothly.

How to install HP Insight Management Agents or WBEM to Windows Server 2008 R2

SYMPTOMS:

While HP doesn’t support Microsoft Windows Server 2008 R2 as operating system for DL3x0 G4, you can install this software to it as usual. But after installation you find out no information at “HP System Management Homepage”.

CAUSE:

This happens because neither WBEM components, nor HP Insight Management Agents cannot be installed.

When you try to install WBEM/HPIMA manually, you receive following error:
Installation for “HP Insight Management Agents for Windows Server 2003/2008 x64 Editions” requires one or more of the following that is not currently installed or in the install set:

– HP ProLiant Advanced System Management Controller Driver for Windows
– HP ProLiant iLO Advanced and Enhanced System Management Controller Driver for Windows
– HP ProLiant iLO 2 Management Controller Driver for Windows
– HP ProLiant iLO 3 Management Controller Driver for Windows

RESOLUTION:

  1. Download HP ProLiant iLO Advanced and Enhanced System Management Controller Driver for Windows Server 2008 x64 Editions (cp010914.exe) to the server.
  2. Extract downloaded file with integrated extract feature.
  3. Set the compatibility mode for cpqsetup.exe as “Windows Server 2008 (Service Pack 1)”.
  4. Run cpqsetup.exe, installation should works fine.
  5. Install HP Insight Management Agents/WBEM as usual.
  6. Also, one unknown device will disappear from Device Manager — it is called “HP ProLiant iLO2 Advanced System Management Controller” from now.