Use the GetProductInfo function to retrieve the product information for the current operating system. If Server Core is running, one of three values will be returned by GetProductInfo: PRODUCT_DATACENTER_SERVER_CORE, PRODUCT_ENTERPRISE_SERVER_CORE, or PRODUCT_STANDARD_SERVER_CORE. Calling GetProductInfo is the recommended way to test if Server Core is running.
Another way to test if Server Core is running is to check the registry under HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion for the value of InstallationType. If it is set to "Server Core", then the Server Core installation option is installed.