- | rssFeed | My book on MSBuild and Team Build | Archives and Categories Wednesday, August 26, 2009

Speaking at St. Louis Day of .NET

On Friday August 28 and Saturday August 29 there will be a conference in St. Louis called St. Louis Day of .NET that I will be speaking at. Here is the outline

Simplify build and deployment of ASP.NET sites with Web Deployment Projects

When you are creating ASP.NET sites, (either ASP.NET web site / ASP.NET Web Project / ASP.NET MVC Project / etc) you will need to deploy the site to machines which will host them. Classically there were two options; xcopy the actual source to the live server and let them be compiled on demand or you could use the aspnet_compiler.exe and aspnet_merge.exe tools to pre-compile the website for you. The little known, yet extremely useful, Web Deployment Projects (an add on for Visual Studio) can greatly simplify the process of build and deployment. Web Deployment Projects will take care of the complexities of the aspnet_compiler.exe and the aspnet_merge.exe tool for you by way of a tight UI integration into Visual Studio itself. Aslo Web Deployment Projects are MSBuild files so you can extend and customize the process to suit your exact needs. In this session we will introduce Web Deployment Project, show that you can perform very powerful actions without writing a single line of code (or even text)! We will also dive into the MSBuild file that is the Web Deployment Project and show how to customize the process.

This session talks about Web Deployment Projects (2005 version) and how they can help in the build and deployment process. If you are going to be attending and interested in build and deployment of ASP.NET web sites and projects then you don't want to miss this session.

Sayed Ibrahim Hashimi

Deployment | msbuild | speaking | Web Deployment Projects Wednesday, August 26, 2009 4:19:16 AM (GMT Daylight Time, UTC+01:00)  #     | 
Friday, August 21, 2009

MSBuild: Building 64 bit versus 32 bit

The other day I had a reader ask me about the difference between 32 and 64 bit builds. Here is his email to me.

I have your Using MSBuild book and I have been learning much with it. One aspect that I cannot seem to find online or in the book is that of the build server environment. Specifically how the choice of Server OS affects your builds and subsequent testing environments. The aspects of most of our build environments are TFS2008 and MSbuild servers running on Server 2003 32bit Standard some have VSTS Team Test edition 2008.

Here are some questions I am looking for answers on.

If my dev team is writing an app for run on 64 bit servers and they will be running VS Team test 2008 on the same build server then do they want VS 2008 64-bit installed as their build server? What could the pitfalls be if they are running builds and tests or just build for that matter on Server 2008 32-bit or 2003 32-bit? Or even just running builds?

If a team is developing an app for 32 bit server 2003 but your build machine is Server2008 64-bit, what could go wrong there? Also in many of the many combinations here are there any false hopes, such as building 32bit on a 64 machine may work but…

Additionally what is to be expected of WOW in this context 64-bit will run Windows on Windows on some apps I think but is this a best practice?

There are so many combinations of the way that an environment could be setup. A blog post exploring these different setups would be great if you could manage it. I just want to make sure my teams are getting what they need and are getting the expected results. Thanks much for your time.

I don't have too much knowledge in this area myself so I asked a couple other guys (William Bartholomew, JB Brown, Jim Lamb, and Grant Holliday) and here is some related info.

Assuming this is a .NET application the bitness of the build machine has no effect on the bitness of the produced application, this is controlled entirely by the project's compilation settings. Basically:

1. If the assembly is configured for Any CPU then it will run as x64 on a 64-bit machine and as x86 on a 32-bit machine.

2. If the assembly is configured for x86 then it will run as WOW64 on a 64-bit machine (i.e. a 32-bit process) and as x86 on a 32-bit machine.

3. If the assembly is configured for x64 then it will run as x64 on a 64-bit machine and will fail to run on a 32-bit machine. For some related info on this see Visual Studio: Why is there no 64 bit version? (yet)

The bitness of a process is determined by the compilation settings for the entry point (i.e. the executable)... You need to ensure that dependencies are compiled with an appropriate bitness for their host process. For example, devenv.exe always runs as a 32-bit process so if you create a Visual Studio add-in that's compiled for 64-bit only then it will never be loadable by Visual Studio. Basically it comes down to that a process can always load a dependency that's compiled as Any CPU or the same bitness as the process.

A few extra points about deployment and runtime.

4 - Extra considerations need to be made if you are dealing with multiple web sites under IIS 6. Since IIS 6 is a single process it is the entry point and determines the 64 bitness of the process it can only be in 1 "bitness" - so all sites under IIS have to be compatible with that bitness choice - IE - no running a 32 bit website and a 64 bit website under the same IIS 6 process. For more info on this see How to switch between the 32-bit versions of ASP.NET 1.1 and the 64-bit version of ASP.NET 2.0 on a 64-bit version of Windows.

5. While a 64 bit entry point/process may load a dependency that is compiled in Any CPU or x64 - it doesn't guarantee that those dependencies are run-time 64 bit process compatible. For example they may use COM-interop which will pass all compile time checks but fail at runtime. You'll need to do your due-diligence here on COM interop and 3rd party assemblies being used.

6. If your assemblies need regsrv32'd and run as a 32 bit WOW process remember that you need to use the 32 bit version of regsvr32.exe to get it registered into the right hive on that 64 bit OS. 

7. There is a similar thing to #6 for installutil.exe - a 64 bit version and a 32 bit WOW version

 

 

Sayed Ibrahim Hashimi

msbuild Friday, August 21, 2009 3:58:27 AM (GMT Daylight Time, UTC+01:00)  #     | 
Friday, August 14, 2009

Web Deployment Projects Slowing Down Builds

I am a pretty big fan of Web Deployment Projects (2005 version) I try and promote their use whenever I get a chance. The other day I had a customer tell me that he added a WDP to his solution and now all the devs were complaining because the build was taking much longer than it was previously. The reason behind this is pretty simple, when you build in Visual Studio it will build the WDP will call the aspnet_compiler.exe and the aspnet_merge.exe tool on your site(s) which can be a lengthy process. When you add a WDP to your solution by default it will be configured to build for each defined configuration. To alleviate this headache you should go to the configuration manager and disable these from building.

Once you open the configuration manager you will be presented a dialog like the following.

Here what you should do is make sure that the Build check box is un-checked for all WDPs. You need to do this for each configuration/platform combination. The idea here is that you want the developers to build inside of Visual Studio as they normally would without any hassles. If developers have to wait for a longer time for a build to complete, not only will it waste their time (which if you add those minutes up becomes a lot of time) but also breeds contempt towards WDPs. Basically they will associate longer build times with WDPs and then they will dislike them. This is the opposite of what you want. You want to encourage them to use WDPs. If the developer needs to build the WDP they can always right click on the WDP and then click build. But better would be to create a build script that can be run on you CI server as well as locally on the developers machine to perform a full build. I am not a fan of using solution files for CI builds, so I would say to create a "master" MSBuild file which uses the MSBuild task to build each project in the correct order and for every desired configuration. If you take this approach then you will never need a WDP to build in your solution. But if you want to use a solution file to build on your CI server then you have at least 2 options.

From these two options I would pick 1, but best would be to create a build which didn't rely on solution files.

 

Sayed Ibrahim Hashimi

Friday, August 14, 2009 4:04:31 AM (GMT Daylight Time, UTC+01:00)  #     | 
Thursday, August 13, 2009

MSBuild: Executing MSTest Unit Tests

I have seen a couple blog entries about executing MSTest unit tests from MSBuild. Most recently I saw the entry by Scott A. Lawrence. So I decided to share how I execute MSTest unit tests from MSBuild

I created a file named Build.Common.UnitTest.targets which contains all the behavior that will execute the test cases. This file can then be imported into whatever scripts that need to execute test cases. The entire file is shown below. We will discuss afterwards.

Build.Common.UnitTest.targets

<?xml version="1.0" encoding="utf-8"?>

<Project ToolsVersion="3.5" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <!-- =======================================================================

    TESTING

  ======================================================================= -->

 

  <!-- Default build settings go here -->

  <PropertyGroup>

    <RunMSTest Condition="'$(RunMSTest)'==''">true</RunMSTest>

    <BuildInParallel Condition="'$(BuildInParallel)'==''">true</BuildInParallel>

  </PropertyGroup>

 

  <Target Name="MSTestValidateSettings">

    <!-- Cleare out these items -->

    <ItemGroup>

      <_RequiredProperties Remove="@(_RequiredProperties)"/>

      <_RequiredItems Remove="@(_RequiredItems)"/>

    </ItemGroup>

   

    <ItemGroup>

      <_RequiredProperties Include="BuildContribRoot">

        <Value>$(BuildContribRoot)</Value>

      </_RequiredProperties>

      <_RequiredProperties Include="OutputRoot">

        <Value>$(OutputRoot)</Value>

      </_RequiredProperties>

 

      <_RequiredItems Include="MSTestProjects">

        <RequiredValue>@(MSTestProjects)</RequiredValue>

        <RequiredFilePath>%(MSTestProjects.FullPath)</RequiredFilePath>

      </_RequiredItems>

      <_RequiredItems Include="AllConfigurations">

        <RequiredValue>@(AllConfigurations)</RequiredValue>

      </_RequiredItems>

      <_RequiredItems Include="AllConfigurations.Configuration">

        <RequiredValue>%(AllConfigurations.Configuration)</RequiredValue>

      </_RequiredItems>

    </ItemGroup>

   

    <!-- Raise an error if any value in _RequiredProperties is missing -->

    <Error Condition="'%(_RequiredProperties.Value)'==''"

           Text="Missing required property [%(_RequiredProperties.Identity)]"/>

 

    <!-- Raise an error if any value in _RequiredItems is empty -->

    <Error Condition="'%(_RequiredItems.RequiredValue)'==''"

           Text="Missing required item value [%(_RequiredItems.Identity)]" />

 

    <!-- Validate any file/directory that should exist -->

    <Error Condition="'%(_RequiredItems.RequiredFilePath)' != '' and !Exists('%(_RequiredItems.RequiredFilePath)')"

           Text="Unable to find expeceted path [%(_RequiredItems.RequiredFilePath)] on item [%(_RequiredItems.Identity)]" />

  </Target>

 

  <UsingTask

      TaskName="TestToolsTask"

      AssemblyFile="$(BuildContribRoot)TestToolsTask-1.3\Microsoft.VisualStudio.QualityTools.MSBuildTasks.dll"/>

 

  <!-- TODO: Create a ValidateTestSettings target and put it on this list -->

  <PropertyGroup>

    <MSTestDependsOn>

      BuildMSTestProjects;

      BeforeMSTest;

      CoreMSTest;

      AfterMSTest

      $(MSTestDependsOn);

    </MSTestDependsOn>

  </PropertyGroup>

  <Target Name="MSTest" DependsOnTargets="$(MSTestDependsOn)"/>

  <Target Name="BeforeMSTest"/>

  <Target Name="AfterMSTest"/>

  <Target Name="CoreMSTest" Outputs="%(MSTestProjects.Identity)">

    <Message Text="Running MSTest for project [%(MSTestProjects.Identity)]"/>

    <Message Text="MSTestProjects.Directory: %(MSTestProjects.RootDir)%(MSTestProjects.Directory)" />

 

    <PropertyGroup>

      <_CurrentConfig>Debug</_CurrentConfig>

    </PropertyGroup>

 

    <PropertyGroup>

      <_SearchPath>$(OutputRoot)$(_CurrentConfig)\%(MSTestProjects.Filename)\</_SearchPath>

      <_TestContainers></_TestContainers>

    </PropertyGroup>

 

    <TestToolsTask

      SearchPathRoot="%(MSTestProjects.RootDir)%(MSTestProjects.Directory)"

      TestContainers="$(OutputRoot)$(_CurrentConfig)\%(MSTestProjects.Filename)\bin\%(MSTestProjects.Filename).dll"/>

 

    <!-- TODO: Read in the results of the tests and get all failures and report them here -->

  </Target>

 

  <PropertyGroup>

    <BuildMSTestProjectsDependsOn>

      BeforeBuildMSTestProjects;

      CoreBuildMSTestProjects;

      AfterBuildMSTestProjects;

      $(BuildMSTestProjectsDependsOn);

    </BuildMSTestProjectsDependsOn>

  </PropertyGroup>

  <Target Name="BuildMSTestProjects" DependsOnTargets="$(BuildMSTestProjectsDependsOn)"/>

  <Target Name="BeforeBuildMSTestProjects"/>

  <Target Name="AfterBuildMSTestProjects"/>

  <Target Name="CoreBuildMSTestProjects" Outputs="%(AllConfigurations.Configuration)"

          DependsOnTargets="CleanMSTest">   

    <!-- Make sure to do clean build in case test cases were added -->

   

    <PropertyGroup>

      <_CurrentConfig>%(AllConfigurations.Configuration)</_CurrentConfig>

    </PropertyGroup>

 

    <Message Text="Building (MSTestProjects.Identity) %(MSTestProjects.Identity)" Importance="high"/>

   

    <!-- Build the projects here. -->

    <MSBuild Projects="%(MSTestProjects.Identity)"

             Properties="Configuration=$(_CurrentConfig);

                          OutputPath=$(OutputRoot)$(_CurrentConfig)\%(MSTestProjects.Filename)\bin\;

                          BaseIntermediateOutputPath=$(OutputRoot)$(_CurrentConfig)\%(MSTestProjects.Filename)\obj\;

                          GenerateResourceNeverLockTypeAssemblies=true;

                          %(ProjectsToBuild.Properties);

                          $(AllProjectProperties);"

             BuildInParallel="$(BuildInParallel)"

             >

    </MSBuild>

  </Target>

 

 

  <Target Name="CleanMSTest">

 

    <MSBuild Projects="@(MSTestProjects)"

         Targets="Clean"

         Properties="Configuration=$(_CurrentConfig);

                          OutputPath=$(OutputRoot)$(_CurrentConfig)\%(MSTestProjects.Filename)\bin\;

                          BaseIntermediateOutputPath=$(OutputRoot)$(_CurrentConfig)\%(MSTestProjects.Filename)\obj\;

                          GenerateResourceNeverLockTypeAssemblies=true;"

         BuildInParallel="$(BuildInParallel)"

             />

   

  </Target>

 

</Project>

There are five important targets which are described below.

Name

Description

MSTestValidateSettings

This validates that the file was provided the needed data values to perform its task, to run the unit tests. For more info on this technique see my previous entry Elements of Reusable MSBuild Scripts: Validation.

MSTest

This is the target that you would execute to run the test cases. The target itself is empty but it sets up the chain of dependent targets.

CoreMSTest

This is the target which executes the test cases. This is preformed using the TestToolsTask.

BuildMSTestProjects

This target is responsible for building (i.e. compiling) the projects which contain the test cases. You don't have to call this it is called automagically.

CleanMSTest

This target will execute the Clean target for all the test projects defined.

 

If you take a look at the CoreBuildMSTestProjects target you can see that I am batching (Target batching to be specific) it on each defined configuration. This is achieved with the attribute Outputs="%(AllConfigurations.Configuration)". If you are not familiar with batching, see the links at the end of this post for more details, and you can always grab my book for even more detailed info J. Then inside that target I build each project by batching (Task batching) the MSBuild task on each project defined in the MSTestProjects item list.

Then inside the CoreMSTest target I execute the test cases. This target is batched for every value in the MSTestProjects item. As I'm writing this I have noticed that I've hard-coded the value for the configuration used in that target to be Debug with the statement

<PropertyGroup>

  <_CurrentConfig>Debug</_CurrentConfig>

</PropertyGroup>

This shouldn't be hard coded, but passed in. I will leave it as is for now though. Then the TestToolsTask is invoked to execute the test cases.

Now that we have written the re-usable .targets file to execute the test cases we need to create a file which will "feed" it the necessary data values and let it do its magic. I created a sample solution, which you can download at the end of this post, which demonstrates its usage. The solution is named MSTestExample and you can see the files it contains in the screen shot below.

Here I've highlighted the two MSTest projects as well as a couple build files. I've already shown the contents of the Build.Common.UnitTest.targets file. Here is the contents of the Build.MSTestExample.proj file.

Build.MSTestExample.proj

<?xml version="1.0" encoding="utf-8"?>

<Project ToolsVersion="3.5" DefaultTargets="MSTest" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

 

  <!--

  Some properties you may be interested in setting:

    Name              Description

    *****************************************************

    RunMSTest         true/false to run unit tests or not

  -->

 

  <PropertyGroup>

    <Root Condition="'$(Root)'==''">$(MSBuildProjectDirectory)\..\</Root>

 

    <BuildRoot Condition="'$(BuildRoot)'==''">$(Root)Build\</BuildRoot>

    <BuildContribRoot Condition="'$(BuildContribRoot)'==''">$(BuildRoot)Contrib\</BuildContribRoot>

 

    <SourceRoot Condition="'$(SourceRoot)'==''">$(Root)</SourceRoot>

    <BuildArtifactsRoot Condition="'$(BuildArtifactsRoot)'==''">$(BuildRoot)BuildAftifacts\</BuildArtifactsRoot>

    <OutputRoot Condition="'$(OutputRoot)'==''">$(BuildArtifactsRoot)Output\</OutputRoot>

    <TreatWarningsAsErrors Condition=" '$(TreatWarningsAsErrors)'=='' ">true</TreatWarningsAsErrors>

  </PropertyGroup>

 

  <PropertyGroup>

    <CodeAnalysisTreatWarningsAsErrors Condition="'$(CodeAnalysisTreatWarningsAsErrors)'==''">true</CodeAnalysisTreatWarningsAsErrors>

    <BuildInParallel Condition="'$(BuildInParallel)'==''">true</BuildInParallel>

   

    <RunCodeAnalysis Condition="'$(RunCodeAnalysis)'==''">true</RunCodeAnalysis>

    <RunMSTest Condition="'$(RunMSTest)'==''">false</RunMSTest>

    <RunStyleCop Condition="''=='$(RunStyleCop)'">true</RunStyleCop>

  </PropertyGroup>

 

  <!-- Configurations that we want to build for -->

  <ItemGroup>

    <AllConfigurations Include="Debug">

      <Configuration>Debug</Configuration>

    </AllConfigurations>

  </ItemGroup>

 

  <!-- =======================================================================

    MSTest

  ======================================================================= -->

 

  <ItemGroup>

    <MSTestProjects Include="$(SourceRoot)TestProject1\TestProject1.csproj">

    </MSTestProjects>

    <MSTestProjects Include="$(SourceRoot)TestProject2\TestProject2.csproj">

    </MSTestProjects>

  </ItemGroup>

 

  <Import Project="$(BuildRoot)Build.Common.UnitTest.targets"/>

 

  <!-- Inject the MSTest targets into the build -->

  <!--<PropertyGroup Condition="'$(RunMSTest)'=='true'">

    <BuildDependsOn>

      $(BuildDependsOn);

      MSTest;

    </BuildDependsOn>

    <BuildMSTestProjectsDependsOn>

      CoreBuild;

      $(BuildMSTestProjectsDependsOn)

    </BuildMSTestProjectsDependsOn>

  </PropertyGroup>-->

 

</Project>

This file is pretty simple. It just creates some properties, and items and then just imports the Build.Common.UnitTest.targets file to do all the heavy lifting. You will probably notice that there are some properties defined that don't make sense here, like CodeAnalysisTreatWarningsAsErrors, this is because this was taken from a build script which does some other tasks. You can ignore those. To see what properties/items are required for the MSTest just look at the MSTestValidateSettings target. Also in the previous code snippet I showed how you could inject the MSTest target into the build process but it is commented out since there is no real build process in this case.

One thing about this approach that may not be ideal is that it will execute the test cases in one assembly at a time and if there is a failure it will not go on to the other assemblies. In my case this is OK because this is for local builds, public builds are executing test cases by Team Build, but this can be looked at.

Questions? Comments?

 

MSTestExamples.zip

MSBuild Batching Links


Sayed Ibrahim Hashim

batching | msbuild | Visual Studio | Visual Studio 2008 Thursday, August 13, 2009 5:11:52 AM (GMT Daylight Time, UTC+01:00)  #     | 
Wednesday, August 05, 2009

I’m featured on dnrtv.com

A few weeks ago Carl Franklin interviewed me for his site dnrtv.com on the topic of MSBuild. I'm on the home page right now but the permalink is http://www.dnrtv.com/default.aspx?showNum=145. Check it out. I think we may do a couple other shows to cover a bit more detail; we just got started with MSBuild in that video.

Sayed Ibrahim Hashimi

dnrTV | msbuild | Video Wednesday, August 05, 2009 4:27:35 AM (GMT Daylight Time, UTC+01:00)  #     |