Category Archives: Windows Server 2012

PSA: Meltdown Patches (CVE-2017-5715, CVE-2017-5754) Could Cause Problems With Hyper-V Live Migration

Suppose you have two Hyper-V servers: On the first server (Server A) you installed both 2018-01 Rollup Update and an updated BIOS release. On the second server (Server B) you installed only the Rollup Update. You added FeatureSettingsOverride, FeatureSettingsOverrideMask, and MinVmVersionForCpuBasedMitigations registry keys to both hosts. Then you rebooted both machines.

On Server A you have a virtual machine. That VM was (re-)booted on that server after the BIOS update and the Rollup Update were installed. Get-SpeculationControlSettings shows that all mitigations are enabled for the machine.
You try to live-migrate the virtual machine from Server A to Server B.

In that case, live migration will complete successfully, but the VM will freeze and won’t be available neither via network, nor via the Hyper-V console.

Solution:

To resume normal VM work, you should either to:

  • Move the VM back from Server B to Server A. It should un-freeze automatically.
  • Forcefully restart the VM using Hyper-V Management snap-in or PowerShell cmdlets at Server B.

I tested it with Windows Server 2012 R2 only, the VM was Windows Server 2016. Not sure if it applies to Server 2016 hypervisors.

AuthenticationSilo claim is not issued

You setup an Active Directory Authentication Policy and use a membership in Authentication Policy Silo as an access control condition. Next you setup Authentication Policy Silo to use the abovementioned Authentication Policy for appropriate principal types. You set the silo into “audit-only” mode.

In that case, AuthenticationSilo claim is not issued for your security principals.

Why does this happen?

As described in 3.1.1.11.2.18 GetAuthSiloClaim section of Active Directory Technical Specification, AuthenticationSilo claim is issued only when policies in Authentication Silo are enforced:
/*
Check if user is assigned to an enforced silo.
*/
assignedSilo := pADPrincipal!msDS-AssignedAuthNPolicySilo
if (assignedSilo = NULL ||
assignedSilo!msDS-AuthNPolicySiloEnforced = FALSE)
return NULL
endif

Resolution

I’ve found no option to modify this behavior yet. Just keep it in mind while you are testing your Authentication Policies configuration.

«0x80070721 A security package specific error occurred» while using WMI between domains

Symptoms:

You have two IBM System x servers with Windows Server 2012 or newer and with «IBM USB Remote NDIS Network Device» network interface enabled. Both of these servers reside in the same AD DS forest but in different domains. You try to setup a WMI session from one to another (using wbemtest, for example). In that case WMI connection fails and you receive an error:

Number: 0x80070721 Facility: Win32 Description: A security package specific error occurred.»
In System log of the server which initiates the connection we have:
Log Name: System
Source: Microsoft-Windows-Security-Kerberos
Date: 10/6/2013 9:13:09 PM
Event ID: 4
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: SRV1.alpha.example.com
Description:
The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server srv1$. The target name used was host/srv2.beta.example.com. This indicates that the target server failed to decrypt the ticket provided by the client. This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using. Ensure that the target SPN is only registered on the account used by the server. This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service. Ensure that the service on the server and the KDC are both configured to use the same password. If the server name is not fully qualified, and the target domain (ALPHA.EXAMPLE.COM) is different from the client domain (BETA.EXAMPLE.COM), check if there are identically named server accounts in these two domains, or use the fully-qualified name to identify the server.

Why does this happen?

Starting from Windows Server 2012, when one machine connects to another computer’s WMI, Windows asks remote system for IP-addresses of its network interfaces. Then operating system chooses which one of them serves its needs to connect best.
In case of IBM System x, both servers have network interface with the same IP-address – 169.254.95.120. Windows chooses this IP-address as the best and tries to connect to it. But instead of remote system, it connects to itself and you see the error.

When you try to connect with wbemtest, it calls a WMI API. In the background WMI uses DCOM to communicate with other servers. When DCOM establishes a session it uses «ServerAlive2» query to check the other server.
Here is what network capture looks like:

14 14:38:55 10.01.2014 0.0111483 srv2.beta.example.com 135 (0x87) 91.103.70.14 55535 (0xD8EF) DCOM DCOM:IObjectExporter:ServerAlive2 Response {MSRPC:10, TCP:9, IPv4:8}
NetworkAddress: srv1
NetworkAddress: 169.254.95.120
NetworkAddress: 192.0.2.1

Same traffic when we access from another end:

38 14:37:12 10.01.2014 37.1871388 srv1.alpha.example.com 135 (0x87) 10.253.12.2 59278 (0xE78E) DCOM DCOM:IObjectExporter:ServerAlive2 Response {MSRPC:14, TCP:13, IPv4:8}
NetworkAddress: srv2
NetworkAddress: 169.254.95.120
NetworkAddress: 203.0.113.2

The application layer chooses the common IP-address and loops back the real connection. The DCOM gets a Kerberos ticket, and tries to authenticate with it, so this is why we see AP_ERR_MODIFIED errors from Kerberos. So a DCOM based communication (for example WMI) won’t work if the participants has a common IP address.

This problem is known by Microsoft and will not be fixed since it is by design.

How to mitigate it?

  1. Just disable IMM USB Network Interface at one or at both servers. But beware: updating of IMM firmware from Windows or using ASU64 will enable this interface again. If you choose this option, I suggest you to setup monitoring to alert you when that interface leave enabled.
  2. Change IP-address at one if the interfaces to something else. You can use almost any private address but here are recommendations from IBM. You even can use DHCP-server to achieve it (I hope you are using separate VLAN for management interfaces, aren’t you?).