I first developed SlowCheetah around VS2010 SP2 with the idea that I could at some point transition this to the appropriate team(s) at Microsoft. Unfortunately I haven't been able to do that, and the existence of this extension has actually worked against that goal.
I'm really happy that SlowCheetah has gotten the attention and success that it has, but now it's time for me to move on.
I will not be adding any features to SlowCheetah myself. If anyone want's to add any features I will try and help guide anyone who is interested.
No fixes for regressions
If there are any scenarios that work in VS2013 RTM that do not work in future version of Visual Studio then I will not be fixing them.
I hope you all understand my situation here. I have spent countless hours working on SlowCheetah and there is very little ROI for me so, I need to move on to focus on other OSS projects that I'm involved in.
Thanks for all the love. I still love SlowCheetah too and I'm sad to see there won't be support for transform in VS "14"
If you haven’t heard I’m working on a project with Mads Kristensen called SideWaffle. SideWaffle is a Visual Studio Extension which contains many different Visual Studio Item and Project Templates. This is a community effort and all open source at https://github.com/ligershark/template-builder. You can create your own Item Templates and Project Templates and send a Pull Request for them to be included in the main repo. Check out the video below for more info on SideWaffle.
SideWaffle intro video
Item Templates are used by VS developers to create files using the Add New Item dialog. SideWaffle already contains a number of Item Templates such as; Angular Controller, robots.txt, SignalR Hub and Client, etc. For more info on how to create Item Templates with SideWaffle watch the 4 minute video below.
Project Templates are the items that show up in the Add New Project dialog. They provide a starting point for developers creating new projects. SideWaffle already has a few project templates as well, such as a Google Chrome Extension. You can learn more about how to create Project Templates in this video.
Now that we’ve gotten the intro out of the way, let’s explore how you can create your own SideWaffle.
How to create your own SideWaffle
The idea behind SideWaffle is that we will have a shared VS extension for popular VS Item and Project Templates. Instead of contributing to the main SideWaffle project you may be interested in creating your own distribution that does not have the standard templates. For example, I’ve heard from both the Orchard and Umbraco that they are interested in creating template packs for their communities. It wouldn’t make much sense to include those templates in the main SideWaffle project. Instead it would be best to create a separate distribution for each; OrchardWaffle and UmbracoWaffle.
So how can you do this? It’s pretty easy actually. SideWaffle is built on top of a NuGet package, TemplateBuilder, which is also open source at https://github.com/ligershark/template-builder. All the core functionality of SideWaffle is contained in that NuGet package. To create your own SideWaffle follow these steps:
If you don’t have it already download and install the Visual Studio SDK, here is 2012 version
Create a new Visual Studio Package project
From the package manager console execute Install-Package TemplateBuilder –pre
After you add the TemplateBuilder NuGet package a few things happen:
The build process of the project is modified to support building Item and Project templates
Your .vsixmanifest file is updated with two new Asset tags
An ItemTemplates folder is created with a sample Item Template
From here on you can build the project and after installing the generated .vsix you can have users easily create instances of your item or project templates.
You can add additional Item Templates, as well as create Project Templates in your project. That’s pretty much all there is to getting started with your own Waffle pack.
When automating web publishing for Visual Studio projects in many cases your first step will be to create a publish profile for the project in VS. There are many cases in which you cannot, or would not like to do this. In this post I’ll show you how you can take an existing project and use an MSBuild file to drive the publish process. In that I’ll also show how you can extend the publish process without modifying either the project or any of its contents.
In this snippet we are passing in a handful of properties. VisualStudioVersion dictates which version of MSBuild targets are used during the build. See http://sedodream.com/2012/08/19/VisualStudioProjectCompatabilityAndVisualStudioVersion.aspx for more details on that. DeployOnBuild=true injects the publish process at the end of build. PublishProfile can either be the name of a publish profile which the project contains or it can be the full path to a publish profile. We will use PublishProfile with that second option, the full path.
So we need to pass in the full path to a publish profile, which typically is a .pubxml file. A publish profile is just an MSBuild file. When you pass in PublishProfile and DeployOnBuild=true, then the publish profile is Imported into the build/publish process. It will supply the publish properties needed to perform the publish.
Let’s see how that works. I have a sample project, MySite, which does not have any publish profiles created for it. I have created a publish profile, ToFileSys.pubxml, in another folder that will be used though. The contents of that file are below.
This publish profile will publish to a local folder. I just created this file in VS with a different project and then just copied it to the folder that I needed, and removed properties which are only used for the inside of VS experience. We can publish the MySite project using this profile with the command below.
When you execute this the file specified in PublishProfile will be included into the build process.
Taking it up a level
Now let’s see how we can take this to the next level by having a single script that will be used to publish more than one project using this technique.
In the sample files (which you can find links for at the end of the post). I have a solution with two web projects, MySite and MyOtherSite. Neither of these projects have any publish profiles created. I have created a script which will build/publish these projects which you can find at build\publish.proj in the samples. The contents of the file are shown below.
publish.proj
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="BuildProjects">
<!--
This file is used in two ways.
1. Drive the build and publish process
2. It is used by the publish process during the build of MySite to configure/extend publish
Note: 1. Is kicked off by the use on the cmd line/build server. 2. Is invoked by this script itself.
This file is injected into the publish process via the PublishProfile property.
-->
<PropertyGroup>
<VisualStudioVersion Condition=" '$(VisualStudioVersion)'=='' ">11.0</VisualStudioVersion>
<Configuration Condition=" '$(Configuration)'=='' ">Release</Configuration>
<!-- Location for build output of the project -->
<OutputRoot Condition=" '$(OutputRoot)'=='' ">$(MSBuildThisFileDirectory)..\BuildOutput\</OutputRoot>
<!-- Root for the publish output -->
<PublishFolder Condition=" '$(PublishFolder)'==''">C:\temp\Publish\Output\</PublishFolder>
</PropertyGroup>
<ItemGroup>
<ProjectsToBuild Include="$(MSBuildThisFileDirectory)..\MySite\MySite.csproj">
<AdditionalProperties>
VisualStudioVersion=$(VisualStudioVersion);
Configuration=$(Configuration);
OutputPath=$(OutputRoot);
WebPublishMethod=FileSystem;
publishUrl=$(PublishFolder)MySite\;
DeployOnBuild=true;
DeployTarget=WebPublish;
PublishProfile=$(MSBuildThisFileFullPath)
</AdditionalProperties>
</ProjectsToBuild>
<ProjectsToBuild Include="$(MSBuildThisFileDirectory)..\MyOtherSite\MyOtherSite.csproj">
<AdditionalProperties>
VisualStudioVersion=$(VisualStudioVersion);
Configuration=$(Configuration);
OutputPath=$(OutputRoot);
WebPublishMethod=FileSystem;
publishUrl=$(PublishFolder)MyOtherSite\;
DeployOnBuild=true;
DeployTarget=WebPublish;
PublishProfile=$(MSBuildThisFileFullPath)
</AdditionalProperties>
</ProjectsToBuild>
</ItemGroup>
<Target Name="BuildProjects">
<MSBuild Projects="@(ProjectsToBuild)" />
</Target>
<!-- ***************************************************************************************
The targets below will be called during the publish process.
These targets are injected into the publish process for each web project.
These targets will not have access to any new values for properties/items from
the targets above this.
*************************************************************************************** -->
<Target Name="AfterWebPublish" AfterTargets="WebPublish">
<Message Text="Inside AfterWebPublish" Importance="high"/>
</Target>
</Project>
This file is pretty simple, it declares some properties which will be used for the build/publish process. Then it declares the projects to be built with an item list named ProjectsToBuild. When declaring ProjectsToBuild I use the AdditionalProperties metadata to specify MSBuild properties to be used during the build process for each project. Let’s take a closer look at those properties.
I’ll explain all the properties now. VisualStudioVersion, Configuration and OutputPath are all used for the build process. The other properties are related to publishing. If you want to publish from the file system those properties (WebPublishMethod, publishUrl, DeployOnBuild, and DeployTarget) must be set. The most important property there is PublishProfile.
PublishProfile is set to $(MSBuildThisFileFullPath) which is the full path to publish.proj. This will instruct the build process of that project to import publish.proj when its build/publish process is started. It’s important to note that a “new instance” of the file will be imported. What that means is that the imported version of publish.proj won’t have access to any dynamic properties/items created in publish.proj.
The reason why PublishProfile is specified there is so that we can extend the publish process from within publish.proj itself. publish.proj has a target, AfterWebPublish, which will be executed after each project is executed. Let’s see how this works.
We can execute the publish process with the command below.
After executing this command the tail end of the result is shown in the image below.
In the image above you can see that the MyOtherSite project is being publish to the specified location in publish.proj and the AfterWebPublish target is executed as well.
In this post we’ve seen how we can use an MSBuild file as a publish profile, and how to extend the publish process using that same file.
A few weeks ago I blogged about a Visual Studio extension, Farticus, which I’m working on with Mads Kristensen. In that post I described how the default compression of a .vsix (the artifact that is created for a Visual Studio Package) is not as small as it could be. It’s better to get a fully compressed VSIX because when users install the component the download time can be significantly reduced. In that post I described how you could use the Zip task from the MSBuild Extension Pack to have a fully compressed .vsix file. I will now show you how I’ve simplified this.
Since my previous post I’ve created a NuGet package, VsixCompress which simplifies this greatly. If you have an existing Visual Studio package and want to have a fully compressed .vsix file then all you need to do is install the VsixCompress package.
After you install this package the following happens.
NuGet package is downloaded and installed to the packages folder
The project is edited to include an import for a new .targets file
The build process is extended to compress the .vsix file automatically
After installing this package once you build the generated .vsix is much smaller than before. In the default case where you select to create a new C# VS Package the created .vsix is 17kb. After adding VsixCompress the resulting .vsix is only 9kb. That’s almost half the size. That’s all you need to know for local development. If you have a build server setup then there are a couple of additional steps. Let’s go over those now.
Build Server Support
I have blogged before about issues of shipping build updates in NuGet. To briefly summarize, when leveraging NuGet Package Restore you have to be careful if any of those NuGet packages have build updates. When using Package Restore the NuGet packages which contain the imported .targets file(s) are restored after the build starts. What this means is that the .targets files will never be imported (or an old copy is imported in the case the file exists from a previous build). The only way to work around this is to restore the packages before the .sln/.csproj file themselves are built. You can read the full details at http://sedodream.com/2012/12/24/SlowCheetahBuildServerSupportUpdated.aspx. I have a NuGet package, PackageRestore, which can help here. Take a look at my previous post How to simplify shipping build updates in a NuGet package. Now that we’ve discussed all the details that you need let’s discuss what my plans are going forward with this.
Plans going forward
I’m hoping to add the following features over the course of the next few weeks.
Have you ever wanted to use Visual Studio to manage project artifacts but wanted to have a fully custom build process? The recommend way to do this is to build a Custom Visual Studio Project System, but there is a much easier way for lightweight needs. In this post I’ll show you how to take an existing project and “replace” the build process used? For example, wouldn’t it be cool if you could develop a Chrome Extension with VS? When you do a build it would be great to generate the .zip file to for the Chrome Gallery in the output folder. Doing this is way easier than you might think.
Before I go over the steps to do this let me explain the “contract” between Visual Studio and MSBuild. The primary interactions around build in Visual Studio include the following actions in VS. Including what VS does when the action is invoked.
Build – VS invokes the DefaultTarget for the project file
Rebuild – VS invokes the “Rebuild” target
Clean – VS invokes the “Clean” target
For more details you can read Visual Studio Integration (MSBuild). The easiest way to completely customize the VS build process is to do the following:
Create the correct project based on the artifacts you plan on using (i.e. if you’re going to be editing .js files then create a web project)
Edit the project file to not import any of the .targets files
Define the following targets; Build, Rebuild and Clean
After that when you invoke the actions inside of VS the correct action will be executed in your project.
Let’s look at a concrete example. I’ve been working on a Chrome extension with Mads Kristensen the past few days. The entire project is available on github at https://github.com/ligershark/BestPracticesChromeExtension. When it came time to try out the extension or to publish it to the Chrome store we’d have to manually create it, which was annoying. We were using a standard web project to begin with. Here is how we made the workflow simple. Edited the project file to disable the Import statements. See below.
For each of these targets I added/updated the condition to ensure that the .targets file would never be loaded. The reason why I did not simply remove these is because in some cases VS may get confused and try to “upgrade” the project and re-insert the Import statements.
After that I added the following statements to the .csproj file.
Here I defined a new property to point to a custom .targets file and a corresponding Import statement. Now let’s take a look at the ligershark.chrome.targets file in its entirety.
As you can see I defined the following targets; Build, Rebuild, and Clean. This is what we discussed earlier as the requirements. Now when I click the Build button in VS a .zip file containing the Chrome Extension is created in the bin\ folder. Additionally no other step (i.e. any typical build step) is performed here. The only thing happening is what is defined in the Build target here. Similarly when I Rebuild or Clean the Rebuild or Clean target is invoked respectively. This is a pretty neat way to modify an existing project to completely replace the build process to suit your needs.
There are a few things in the .targets file that I’d like to point out.
UseHostCompilerIfAvailable
In the .targets file I have set the property declaration <UseHostCompilerIfAvailable>false</UseHostCompilerIfAvailable>. When you are working with a project in Visual Studio there is a compiler, the host compiler, that Visual Studio has which can be used. For performance reasons in most cases this compiler is used. In this case since we do not want any standard build actions to be executed this performance trick is undesired. You can easily disable this by setting this property to false.
After this MSBuild will always be invoked when performing any build related action.
Visible metadata for Item Lists
When using MSBuild you typically represent files as values within an item list. One issue that you may encounter when modifying a project which will be loaded in VS is that the files inside of those item lists will show up in Visual Studio. If you want to disable this for a particular value in an item list you can add the well known metadata <Visible>false</Visible>. For example you could have the following.
If you don’t want to add this to every value in the item list then you can set the default value using an ItemDefinitionGroup. For example take a look at the elements below.
After this declaration is processed if the Visible metadata is accessed for AppFileNameItem or AppFolderItem on a value which does not declare Visible, then false is returned.
The rest of the content of the .targets file is pretty straightforward. This post shows a basic example of how you can take an existing Visual Studio project and completely replace the build process.
When you are building a .sln file or a .csproj/.vbproj/etc you need to be careful how you set the Configuration property. Once this property is set you do not want to change the value of it. Because of this it is best to specify this as a global property. To do that you can pass it in on the command line. If you are using the MSBuild task then you can pass it in via Properties or AdditionalProperties. What you do not want to do is to set this value inside your .csproj/.vbproj or any imported file. The only time when this is OK if there is a condition which will not set the property if it’s already been set. This is exactly how the default .csproj/.vbproj files are authored:
If you do set this property w/o the conditional not to overwrite an existing value you may experience really strange behavior during your builds. And it may be difficult to diagnose the issue as well.
Now let me explain why you cannot do this. When you build a .csproj/.vbproj MSBuild will start creating an in-memory representation of the entire project. It will start with the .csproj/.vbproj file itself. It will read the file from top to bottom. When a property is encountered it is evaluated. If a property is encountered that relies on another one, for example
The properties inside the expression will be evaluated on whatever values exist for the properties at the time. In this case ItermediateOutputPath will be the value of BaseIntermediateOutputPath and Configuration.
If a property is encountered which specifies a value for a property which has been previously declared, the previous value will be discarded.
The implications of this are subtle but very simple; once a property has been set which has dependent properties you must not overwrite that property. This is because when a dependent property is encountered it is evaluated immediately. You cannot re-evaluate that property. So if you set a property during your build, some existing dependent properties which were evaluated before the value change will continue to use the old value. There is no way to re-evaluate those properties.
Now lets see how this relates to the Configuration property specifically.
The contents of the .csproj/.vbproj are properties/items along with an import to Microsoft.csharp.targets (Microsoft.VisualBasic.targets for .vbproj) which then imports Microsoft.Common.targets. If you look in Microsoft.common.targets you will see many different properties which are declared using $(Configuration). So this means that you must treat the Configuration property with care and abide by the rule that I have outlined above.
FYI If you want more details on this I have explained this entire process in great detail in my book Inside MSBuild and Team Build.
Configuration for web publishing
If you have used the new web publish experience in Visual Studio 2012 (also available for VS2010 from the Azure SDK) you may have noticed that the web publish profiles are MSBuild files. They are stored under Properties\PublishProfiles (My Project\PublishProfiles for VB) and have extensions of .pubxml. In that file (or in a .pubxml.user file associated with it) you will find a property defined as
This value corresponds to the value for the Configuration drop down in the VS web publish dialog. When you kick off a publish in VS we use this value and kick off a build and specify Configuration as a global property which is passed in. The reason why we did not name this property Configuration is because the web publish profile is imported into the web project itself during the publish process. By the file being imported you can natively access all the properties/items of the .csproj/.vbproj and also easily extend the build process. Since that is the case we cannot set the Configuration property because we know it’s value has already been set. Due to this when publishing from the command line you must specify the Configuration property.
Web Deploy (aka MSDeploy) uses a provider model and there are a good number of providers available out of the box. To give you an example of some of the providers; when syncing an IIS web application you will use iisApp, for an MSDeploy package you will use package, for a web server webServer, etc. If you want to sync a local folder to a remote IIS path then you can use the contentPath provider. You can also use this provider to sync a folder from one website to another website.
The general idea of what we want to do in this case is to sync a folder from your PC to your IIS website. Calls to msdeploy.exe can be a bit verbose so let’s construct the command one step at at time. We will use the template below.
We use the sync verb to describe what we are trying to do, and then use the contentPath provider for both the source and the dest. Now let’s fill in what those values should be. For the source value you will need to pass in the full path to the folder that you want to sync. In my case the files are at C:\temp\files-to-pub. For the dest value you will give the path to the folder as an IIS path. In my case the website that I’m syncing to is named sayedupdemo so the IIS path that I want to sync is ‘sayedupdemo/files-to-pub’. Now that give us.
For the dest value we have not given any parameters indicating what server those command are supposed to be sent to. We will need to add those parameters. The parameters which typically need to be passed in are.
ComputerName – this is the URL or computer name which will handle the publish operation
Username – the username
Password – the password
AuthType – the authType to be used. Either NTLM or Basic. For WMSvc this is typically Basic, for Remote Agent Service this is NTLM
In my case I’m publishing to a Windows Azure Web Site. So the values that I will use are:
All of these values can be found in the .publishSettings file (can be downloaded from Web Site dashboard from WindowsAzure.com). For the ComputerName value you will need to append the name of your site to get the full URL. In the example above I manually added ?site=sayedupdemo, this is the same name as shown in the Azure portal. So now the command which we have is.
OK we are almost there! In my case I want to make sure that I do not delete any files from the server during this process. So I will also add –enableRule:DoNotDeleteRule. So our command is now.
At this point before I execute this command I’ll first execute it passing –whatif. This will give me a summary of what operations will be without actually causing any changes. When I do this the result is shown in the image below.
After I verified that the changes are all intentional, I removed the –whatif and executed the command. After that the local files were published to the remote server. Now that I have synced the files each publish after this will be result in only changed files being published.
In the case of the question it was asked with dest:auto, you can use that but you will have to pass in the IIS app name as a parameter and it will replace the path to the folder. Below is the command.
When we released VS2010 we add support for web.config (XDT) transforms during publish/package. Note: From now on I’ll only use the word publish from now on but the full content relates to packaging as well. In the original implementation when you published your web project the web.config file would be transformed by the file web.{Configuration}.config, where {Configuration} is the project build configuration. For example Debug, or Release. If you publish on Release and there exists a web.release.config we will take your web.config and transform it with web.release.config before publishing.
Cascading web.config transformations
In VS 2012 (as well as the publishing updates for VS2010 through the Azure SDK) now support the concept of publish specific transforms. You can also now specify the project configuration used for a profile when publishing on the publish dialog.
In this case I have created a profile named Production and set the Configuration to Release. When I publish this project the following transformations will be applied (if the files exist) in this order.
web.release.config
web.production.config
I think that we got this wrong when we initially implemented the support. We should have created profile specific transforms instead of ones based on build config, but having these cascading transforms are still pretty useful. For example I may want to remove the attribute debug=”true” from the compilation element and then inside of the profile specific transform we would override appSettings/WCF endpoints/logging config/etc for that environment.
In VS there is a right-click option on web.config for Add Config Transform, but we were not able to update the functionality of that to automatically create profile specific transforms. Don’t worry it will be released soon with our first set of updates for web tooling. For now you will need to create a new file with the correct name and add it to your project. Note: if you want it to show up nested under web.config you’ll need to add the metadata <DependentUpon>Web.config</DependentUpon> to the item in the .csproj/.vbproj file.
web.config transform preview
Previously the only way to test the functionality for these transformation was to actually publish or package the web project. This gets old pretty quick. In order to simplify creating these transforms we have introduced the Preview Transform menu option. This is the coolest feature in VS 2012 (OK I’m a bit biased, but still its the coolest).
In my web.release.config I have left the default contents, which just removes the debug attribute. Here is what I see when I select this on web.release.config for my project.
You can see that in the image above we can see that the debug flag was indeed removed as expected.
In my web.production.config I have a transform which simply updates the email app setting value. Here is the really cool part when I preview the transform for web.production.config the previewer will look into the profile and determine the build configuration which has been configured, and it will ensure that transform is applied before the profile specific one. For example take a look at the result for web.production.config.
In the image above you can see the note that web.release.config was applied first followed by web.production.config. In the result we can see that web.release.config removed the debug flag and that web.production.config updated the email address value.
We also do a little bit to help out in case there are errors in either the web.config or a transform. You can see errors in the Output Window and double click it to go directly to where the error exists.
One of the most requested features of Visual Studio 2012 was the ability to open projects in both VS 2012 as well as VS 2010 (requires VS 2010 SP1). In case you haven’t heard we did implement that feature. You may be wondering how we were able to do this and how this may impact you.
If you open the .csproj/.vbproj for a Web Project created in VS2010 you will see the following import statement.
When you open this project in VS 2012 there are a few changes made to your project file to ensure that it can be opened in both VS 2010 SP1 and VS 2012. One of the changes made to the project when it is first loaded in VS 2012 is to add the following to replace that import statement.
We removed the hard-coded 10.0 and instead used the property VisualStudioVersion. When building in Visual Studio 2012 this value will always be 11.0, but for VS 2010 it doesn’t exist. That is why we defaulted it to 10.0 above.
There are some scenarios where building from the command line will require to set this property explicitly. Before we get there let me explain how this property gets set (in this order)
If VisualStudioVersion is defined as an environment variable/global MSBuild property, that is used.
This is how VS and the VS developer command prompt set this value
Based on the file format version of the .sln file (toolset used is sln file format –1)
To simplify this statement, the .sln file will build with specifying VisualStudioVersion to the value of the version of VS which created the .sln file.
Choose default
10.0 if VS 2010 is installed
Highest-versioned sub-toolset version installed
For #2 when you are building a .sln file the value of VisulStudioVersion will be –1 of the Format Version found in the .sln file. The important thing to note here is that if you build a .sln file it will build with the value of VisulStudioVersion corresponding to the version of VS which created the .sln file. So if you create a .sln file in VS2012 and you always build that .sln file the value for VisualStudioVersion will be 11.0. In many cases if you build the .sln file you are good.
If you are building .csproj/.vbproj files w/o going through a .sln file? If you build a web project from the command line (not the developer prompt) then the value for VisualStudioVersion used will be 10.0. That is an artifact of the properties which I showed above. In this case you should pass this in as an MSBuild property. For example
In this case I’m passing in the property explicitly. This will always override any other mechanism to determine the value for VisualStudioVersion. If you are using the MSBuild task in a build script, then you can specify the property either in the Properties attribute or the AdditionalProperties attribute. See my previous blog post on the difference between Properties and AdditionalProperties.
If you encounter any funny behavior when building/publishing and you notice that the wrong .targets files are being imported then you may need to specify this property.
I jus released a new version of SlowCheetah, my Visual Studio extension which enables you to transform app.config, and other XML files, during F5 for non-web projects. In this update I added support for Visual Studio 2012 as well as the bug fixes below.
When building Visual Studio 2012 we made an effort to reduce the amount of menu options which are shown on toolbars as well as context menus. If you have used any of the pre-release versions of VS 2012 then you might have noticed that the Build Deployment Package and Package/Publish Settings context menu options are gone from Web Application Project.
In VS2010 when creating a Web Deploy package the publish dialog was not used because there were no relevant settings. Now that we have enabled features like integration to enable Entity Framework Code First migrations, incremental database updates (coming in the final release), connection string updates, etc. we needed to find a way that you could leverage these great features when creating Web Deploy packages. The solution that we decided to go with was to have first class support for creating packages directly from the publish dialog.
After that, we felt that it would be better to have a single way to create a package than two different ways with pros and cons. And we would benefit by being able to simplify our context menu. To create a package for your Web Application Project in VS 2012 (or in VS 2010 if you have the Azure SDK 1.7+) you can follow the steps below.
Right click on your project and select Publish
Create a new profile for your Web Deploy package
Specify the path where the package should go (must end with a .zip)
Click Publish
After the initial create for the profile, creating additional packages is even easier. You just right click on your project select Publish and then click the Publish button.
The reason why we remove the context menu for the Package/Publish Settings is that most of the package and publish related settings are on the publish dialog. The existing settings from VS 2010 are still available on the property pages if you need to get to them.
One other change which we made was to hide the One Click toolbar. This is a toolbar which can be used to publish your web project in one click after the web publish profile has been created. After looking at the number of times that the button was used we determined that it did not meet the bar to be shown by default. Don’t worry if that was your favorite button (I know it was mine), you can bring it back quickly. The easiest way to turn it on is to press CTRL + Q (to bring up the quick launch), type in ‘one click’ and then click on the single result. That should show the One Click toolbar.
The release of Visual Studio 2012 is right around the corner. If you’ve been following our blog/twitter then you may know that many of the Web related components of Visual Studio are now“out-of-band”, meaning that we can update them independently of Visual Studio itself. Because of this we are planning to ship updates a few times a year. We are currently in the process for planning our first update for VS 2012 and we wanted to share some of the items that we are planning to work on. We would like to get your feedback on these ideas to ensure that we are doing the right things. If you have comments please do let us know. FYI the updates that I describe below will be made available for both VS 2012 as well as VS 2010.
Website Project Updates
When you use Visual Studio there are two types of web projects you can create: a Web Application Project (WAP, this includes MVC) and a Website Project (WSP). In VS 2012 we did a lot of work to update the publish experience for WAPs. When we set out for these updates we planned to take these enhancements and make the available to Website projects as well. As we progressed it became evident that we did not have the resources to implement the features for both WAP and WSP in time for VS 2012. So we focused on creating the right experience for WAPs and we also added an extensibility point into WSP which will allow us to replace the existing publish dialog. The reason why we focused on WAP first was because WAPs already have support for Web Deploy (MSDeploy) based publishing as well as a formal build/publish process captured in MSBuild. WSP does not have either of these so it would have taken more time to start with WSP.
When VS 2012 is released the publish experience for WSP will be the same which was available in VS2010 but we will have a release a few months after the release which will bring all the publish goodness to Website Projects! This will include all the features which are available to Web Application Projects including; the ability to publish using.
Enabling Entity Framework CF migrations in web.config during publish
Incremental database schema publish
File preview
etc
Since both project systems will be using the exact same code whenever we add a feature it will immediately be available to both. In today’s world they are two entirely different code bases (the WSP publishing experience is currently native code where as the WAP publish dialog is managed). This will allow us to maintain a consistent publish experience and also enable us to deliver features more quickly.
Since WSP does not have a “build” process you might be wondering how we are planning to hook this all together since our entire publish pipeline is captured in MSBuild. Here is a rough idea of what we are currently thinking. After you use the new publish dialog to create a publish profile for the first time we will do a number of things:
Create a new folder to contain needed artifacts
The folder may be in the root of the Website project, but we are considering placing it somewhere outside as well
The name of this folder is not yet finalized, but it could be something like _Publish or _Project
Create an MSBuild file for the Website in that folder, this will capture information like; included files, references, properties to control publish process
The primary purpose of dropping this file is to facilitate command line publish scenarios, without this we could publish from Visual Studio but not from the command line
When you make changes in Visual Studio like adding a reference we will automatically keep this file up to date
Create the publish profile in that folder (this is a .pubxml file)
When the site is being published the working directory (i.e. obj\ folder) will be outside the root of the website, most likely in the same folder which contains the WSP root
After we have those things in place for the most part the publish dialog will be able to treat both projects in the same way. These files will by default be stored in version control and can be shared with other team members. Sensitive information such as the MSDeploy password will be stored in another file and encrypted per-user/per-machine in a .user file which is not checked in.
Web Deployment Projects
A few months after Visual Studio 2005 shipped we released the first version of Web Deployment Projects, and we updated WDP for both VS 2008 and VS 2010 and released them a few months after each of those releases as well. WDPs are used for a few different scenarios including the following.
Publishing a Website project using MSDeploy
Command line publish support
Customizing the publish process for a WAP
Running ASP.NET precompile/merge for a WSP or WAP
When we looked at the scenarios that WDPs are typically used for and then compared that to features which we have for WAP and WSP (after the updates mentioned above) we noticed that most (if not all) scenarios where WDP is used can be covered without requiring a WDP. Our new publish experience already has first class support for MSDeploy, command line publishing, and for extensibility so that covers #1-#3. Regarding #4 for WAPs we have already added ASP.NET precompile/merge as a publish option (you can find it in the Package/Publish Web tab). Since WSP does not have any property pages we are likely to move that option to being on the publish dialog itself or we will expose those options in another way for WSP. We have not yet settled on that design.
It may not seem like it but updating WDP to support VS2012 is a significant effort. Knowing that, and that most of the scenarios where WDP are used can now be transitioned to the new experience, we have decided to invest in giving WSP projects first class publishing support instead of updating VS 2012 to support WDP. We think that this is the right decision because we are unifying the publish experience for both project systems and it allows us to create a deeper set of features going forward instead of investing in different things and wasting effort. If you have a scenario that you think we have missed please do not hesitate to reach out to me by email at sayedha@microsoft.com and let me know.
If you have been following our blog and twitter accounts then you most likely have heard that some of the Web components of Visual Studio 2012 are now “Out of Band”. What that means is that we can update those components separately from Visual Studio itself. Because of this we will be posting updates on a regular basis. Today marks the first of these updates, and its targeting Visual Studio 2012 RC. In this post I will describe how to get the update and then what the update contains.
How to get the update
If you already have Visual Studio 2012 RC installed when you launch VS you will see a notification in the task tray like the image below.
Note: VS checks for updates once a day so if you don’t see this notification today its likely that the check has already been performed
After you click on the notification you will be brought to the Extension and Updates dialog. To get the web updates you should go to the Visual Studio Gallery tab and then click Update on the Web Tooling Extensions item.
After installing the update you will need to restart VS.
What is contained in the update
The goals that we had when creating this update mainly consisted of.
Fix key customer reported bugs which we didn’t have time to complete for the RC release
Test out the update mechanism publicly
Since the intent of this release was not to light up any new features you won’t find any new functionality, but you may discover that an issue you are running into has been fixed. Most of the bugs which we fixed were filed by customer on Microsoft Connect, though some came in through other channels. For the Connect bugs you will see a link to the bug details, for Connect bugs filed as private there will not be a link. Below you will find a list of the fixed bugs as well as a brief description of the bug.
Web publishing raises an exception due to <connectionStrings> content
If the root web.config contained elements under <connectionStrings> that did not have a name attribute an exception was raised.
Reported Bugs
Error while trying to publish ASP.NET Web API project (Connect private)
"Exception in Executing Publishing" when Build > Publish Selection (Connect private)
VS2012 RC consistently crashes when publishing WCF service to file share (Connect private)
Web publish doesn’t include all files if there is no project configuration matching the solution configuration
The drop down for Configuration on the publish dialog was mapped to Solution Configurations instead of Project Configurations. Because of this if a Solution Configuration was selected which did not have a corresponding Project Configuration the files would not be published. We have updated the dialog to show the correct value.
Reported Bugs
File export for web application not working properly (Connect private)
One-click Web Deployment using FTP does not work (Connect private)
Wrap up
We are really excited to be able to publish updates on a more regular basis and we hope that you will find that helpful as well. In case you guys were wondering if we listen to feedback/Connect bugs I hope that this post helps to show that we are listening to feedback and working to resolve bugs which get filed on Connect. Keep filing the bugs so that we can make our product even better. If you have already filed bugs on Connect then Thank You!
I have written a few posts recently describing out updated web publish experience. These new experience is available for both Visual Studio 2010 as well as Visual Studio 2012 RC. You can use the links below to download these updates in the Azure SDK download. Below are links for both versions.
We have two sets of ASP.NET providers which currently exist; the ASP.NET SQL providers, and the ASP.NET Universal Providers. In VS 2010 the SQL providers were in only providers used for our project templates. In VS 2012 we have switched to using the Universal Providers. One of the drawbacks of the SQL providers is that it leverages DB objects of SQL server which are not available in SQL Azure.
In our updated web publish experience we have an Update Database checkbox which can be used to incrementally publish the database to the destination database. In this case if the source connection string is used by the ASP.NET SQL providers and you are publishing to SQL Azure then you will see the following message on the dialog.
The publish dialog is letting you know that the SQL providers are not compatible with SQL Azure and helps you convert to using the Universal Providers. After you install the Universal Providers the web.config entry will be commented out and new entries will be inserted for the Universal Providers. Your existing database will not be impacted, we’ll create a new connection string pointing to a new database. If you had any data in the SQL Providers database you will have to re-create those objects in the new database.
If you have any questions please feel free to directly reach out to me at sayedha(at){MicrosoftDOTCom}.
In Visual Studio 2010 we introduced a database publishing experience in the Package/Publish SQL (PP/SQL) properties page. This support relies on generating create scripts from the source database and then executing those scripts when you publish your web application. For more details regarding this feature take a look at Package/Publish SQL Tab, Project Properties. One of the negative aspects of the implementation that we have there is that the DB schema publishing is not incremental, it always executes create scripts. So in many cases you publish your app for the first time and things are great, but the next time you try to publish you receive errors because it tries to create DB objects which already exist.
In our improved publish dialog we have a new database publishing experience which is incremental, so you don’t run into the same issues when re-deploying your database. All of the configuration for this is within the publish dialog itself. If you have a project which already has DB publish settings on the Package/Publish SQL tab (under any build configuration/platform) then you will see the message shown in the image below.
The new incremental DB schema publish mechanism is not compatibly with the feature set on the PP/SQL tab. For each web publish profile you can either use the settings on the PP/SQL tab or you can opt-in to the new experience on the publish dialog. We implemented it this way in order to introduce the new feature set and to ensure that existing publish scenarios are not broken.
FYI there is a related post on why the Update database checkbox is disabled in the publish dialog which you may be interested in.
If you have any questions please feel free to directly reach out to me at sayedha(at){MicrosoftDOTCom}.
If you have tried out our new Web Publish experience in Visual Studio you may have noticed that the Update Database checkbox is disabled. See the image below.
The intended behavior of this checkbox is to enable you to incrementally publish your database schema from the source (the connection string in web.config) to the destination (whatever connection string is in the text box). The difference between an incremental publish and a typical publish is that for incremental publishes only changes are transferred from source to destination. With a full publish the first time that you publish your DB schema everything is created, and the next time that you try to publish you will receive an error because it tries to re-create existing DB objects.
The functionality of the Update database checkbox leverages an MSDeploy provider. We were hoping to complete that provider and give it to hosters in time for the release but we were unable to do so. We are working on completing the provider and partnering with hosters to install these in time for the launch of Visual Studio 2012 RTM.
In the mean time if you need to publish your DB schema you can use the Package/Publish SQL tab (caution: the DB publishing here is not incremental). If you are going to use the PP/Sql tab to publish to SQL Azure then there are some special consideraions that you will need to take. You can learn more about those by visiting http://msdn.microsoft.com/en-us/library/dd465343.aspx and searching for “Azure” on that page.
If you have any questions please feel free to directly reach out to me at sayedha(at){MicrosoftDOTCom}.
This is a tricky problem to solve for the general case, but it is pretty easy in your case because CoreCompile has special built in support for this scenario. Before I go into the details of how you can accomplish this with CoreCompile let me explain how it works in general.
Explanation of the general case
In MSBuild targets are skipped due to Incremental Building. Incremental Build is driven purely off of the Inputs and Outputs attributes on the Target itself. The inputs are a list of files that the target will "use" and the outputs are a list of files that are "generated" by the target. I'm using quotes because its a loose concept not a concrete one. To simplify it you can just treat inputs/outputs as lists of files. When the target is about to be executed MSBuild will take the inputs and compare the timestamps of them to the outputs. If all the outputs are newer then the inputs then the target will be skipped. (FYI If you want to know what happens when only some outputs are out-of-date read my blog athttp://sedodream.com/2010/09/23/MSBuildYouveHeardOfIncrementalBuildingButHaveYouHeardOfPartialBuilding.aspx).
In any case if you want a target to be skipped you have to craft your Inputs/Outputs correctly. In your case you want to skip your target whenever the CoreCompile is skipped, so at the surface it would seem that you could simply copy the Inputs/Outputs of CoreCompile but that doesn't work. It doesn't work because when CoreCompile is executed the files may be out-of-date but that target itself brings them up-to-date. Then when you target is executed since they are all up-to-date it will be skipped. You would have to copy the Inputs/Outputs and append an additional file to inputs/outputs which you target creates. This would ensure that your target wouldn't get skipped during that first pass.
Specific solution for CoreCompile
If you take a look at the project file you will see towards the bottom that the file Microsoft.Common.targets is Imported, this file will then import the language specific .targets file. For example it will Import either Microsoft.CSharp.targets or Microsoft.VisualBasic.targets (if you are using C# or VB). In those .targets files you will find CoreCompile defined. In the definition for CoreCompile you will find the following at the end.
This will call all the targets defined in the TargetsTriggeredByCompilation property. So if you want your target to be called whenever CoreCompile is executed you can extend that property. Here is how to do that.
In this case I define the property TargetsTriggeredByCompilation and I append MyCustomTarget to it. It's very important that you include the $(TargetsTriggeredByCompilation); there, if you don't then you won't be appending but overwriting. So if anyone else used this technique you'd wipe out their target.
Below is an image showing where I build once and CoreCompile and MyCustomTarget are executed. Then the second build CoreCompile is skipped any MyCustomTarget is never called.
If you have used the Visual Studio web publish in either VS 2010 or VS 11 to create Web Deploy packages then you probably know that we parameterize connection strings in web.config automatically. In case you are not familiar with Web Deploy parameters, they are a way to declare that you want to easily be able to update a value of something when publishing the package later on. Connection strings are good examples of something which typically needs to be updated during publish.
As I previously stated if you create a Web Deploy package in Visual Studio we will automatically create Web Deploy parameters for all your connection strings in web.config. Earlier today I saw a question on StackOverflow asking how to parameterize connection strings in non-web.config files (question actually asked something else, but I think this is what he’s really wanting). I created a sample showing how to do this. Below is what the connectionStrings element looks like in web.config.
In order to parameterize these connection strings you will have to extend the Web Publish Pipeline. To do that create a file named {project-name}.wpp.targets in the root of the project in which you are working (for VS 11 projects you can place all this directly inside of the .pubxml files). This will be an MSBuild file which will get imported into the build/publish process. Below is the file which needs to be created.
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<!-- Here we need to declare MSDeploy parameters for connection strings in connectionStrings.config -->
<MsDeployDeclareParameters Include="ApplicationServices-ConnectionString" >
<Kind>XmlFile</Kind>
<Scope>connectionStrings.config$</Scope>
<Match>/connectionStrings/add[@name='ApplicationServices']/@connectionString</Match>
<Description>Connection string for ApplicationServices</Description>
<DefaultValue>data source=(localhost);Initial Catalog=AppServices</DefaultValue>
<Tags>SqlConnectionString</Tags>
</MsDeployDeclareParameters>
<MsDeployDeclareParameters Include="OtherConnectionString-ConnectionString" >
<Kind>XmlFile</Kind>
<Scope>connectionStrings.config$</Scope>
<Match>/connectionStrings/add[@name='OtherConnectionString']/@connectionString</Match>
<Description>Connection string for OtherConnectionString</Description>
<DefaultValue>data source=(localhost);Initial Catalog=OtherDb</DefaultValue>
<Tags>SqlConnectionString</Tags>
</MsDeployDeclareParameters>
</ItemGroup>
</Project>
Here you can see that I am creating values for MSDeployDeclareParameters. When you package/publish this item list is used to create the MSDeploy parameters. Below is an explanation of the metadata values each contain.
Kind = for this case it will always be Xmlfile, learn more
Scope = a regular expression to the file which needs to be modified
Match = an XPath expression to the attribute/element to be updated
Description = optional description (this will show up in the IIS manager if the pkg is imported)
DefaultValue = optional default value for the for the parameter
Tags = optional, for connection strings use SqlConnectionString
After you create this file you will need to close/re-open VS (it caches imported .targets files). Then you can create a web deploy package. When you do so these new parameters will be declared. In my case I then imported this in the IIS manager and here is the dialog which shows up for the parameters.
As you can see the Application Path parameter is shown there as well as my custom connection string values. When I update the values in the text box and opened connectionStrings.config on my web server they were the values I entered in the dialog box.
A couple months ago I blogged about a Package-Web which is a NuGet package that extends the web packaging process in Visual Studio to enable you to create a single package which can be published to multiple environments (it captures all of your web.config transforms and has the ability to transform on non-dev machines). Since that release I have updated the project and tonight I created a video which shows the features a bit you can check it out on Youtube. It’s embedded below.
You can install this via NuGet, the package name is PackageWeb.
A few months ago I published a Visual Studio add-in,
SlowCheetah – XML Transforms, which enables you to transform any XML file during a build in the same way that you can transform the web.config during publish/package for web projects. Since then I’ve received numerous requests to demonstrate how the transforms can be kicked in from a CI server. It’s actually pretty easy because all the logic from a transform perspective is contained inside of MSBuild tasks/targets. The Visual Studio add-in itself is only delivering the support for gestures/preview.
Before I dive into how to enable this for a CI server let me explain how the plugin works and then the CI integration will be come much clearer. When you load a project/solution in Visual Studio for the first time after you installed SlowCheetah a set of files will be written to %localappdata%\Microsoft\MSBuild\SlowCheetah\v1. Those file are:
Name
Description
Install-Manifest.xml
An XML file which describes the files that are installed. This is used by the plugin itself, you should never have to do anything with this file.
SlowCheetah.Tasks.dll
The assembly which contains the MSBuild task which transforms XML files.
SlowCheetah.Transforms.targets
The MSBuild file which enables XML file transformation
The add-in adds the following menu command to any XML file for supported project types.
When you click on this for the first time the add-in needs to make some changes to your project file so you are prompted to accept that. After you do the following changes are applied to your project file.
A new MSBuild property SlowCheetahTargets is added to the project and it points to the SlowCheetah.transforms.targets file in %localappdata%
An import for that file is added
More specifically the elements added to your project file are.
Note: If you look at your project file they will not be next to each other. The property will be towards the top of the project file and the import towards the bottom.
So now you might have an idea of why this won’t work on your CI server, the file SlowCheetah.Transforms.targets will not exist in the %localappdata% folder. There is an easy to way to fix this which doesn’t require any changes to your CI server. Check in the files and update your import. Here is what I did:
Create a folder named SlowCheetah in the same folder as my solution
Added SlowCheetah.Tasks.dll and SlowCheetah.Transforms.targets to that folder
Create a solution folder named SlowCheetah in the solution
Drag and drop the files from your SlowCheetah folder into the SolutionFolder
Updated my project file to point to these files
Now your solution should look roughly like the following image.
Note: Steps #3/#4 are not strictly required
For #5 you’ll have to edit your project file, don’t worry its an easy modification. Just follow these steps:
Right click on the project and select unload
Right click on the unloaded project and select Edit
Update the value for the SlowCheetahTargets import
In my case the folder that I placed the files into is 1 directory above the project itself. So the new value for the property is as follows.
Back in November I participated in Virtual Tech Days which is an online conference presented by Microsoft. In the session I discussed the enhancements to web deployment using Visual Studio 2010 and MSDeploy. Some of the topics which I covered includ:
web.conig (XDT) transforms
How to publish to local file system using Visual Studio
How to publish to a 3rd party host using Visual Studio via MSDeploy
How to publish to local IIS server using the .cmd file generated by Visual Studio
How to use msdeploy.exe to delete IIS applications
How to use the IIS Manager to import web packages
How to use msdeploy.exe to deploy a web package to the local IIS server
How to use msdeploy.exe to deploy a web package to a remove IIS server
How to use msdeploy.exe to deploy a web package & set parameters using SetParameters.xml to a remote IIS server
You can download the video & all of my sample files at http://virtualtechdays.com/pastevents_2010november.aspx. In the samples you will find all of the scripts that I used and a bunch of others which I didn’t have time to cover. Enjoy!
Do you have a great idea for the next version of Visual Studio and you want to get your voice heard? Now is your chance! Go to twitter and post a message stating your request with the hash tag #vswish and we will take a look at it. We are listening to that channel so please do submit your feedback.
Have you ever executed a .cmd file (or .bat) file and received a message that looks like the following at the top of your script (including image for compatibility) ? This was really annoying to me until I find out what was happening. I noticed that Visual Studio (2010 at least, but I think previous version as well but not sure) was changing the encoding of my .cmd files to be UTF-8! For example consider this simple script (sample.cmd) that I created with Notepad.
echo 'hello world'
When I execute this .cmd file from the command line the results are as shown below.
What I did then was to simply create a copy of that file and placed that in a file named sample-vs.cmd and then edited the script using Visual Studio 2010 to have the contents below.
echo 'hello world from Visual Studio'
When I execute that .cmd file to my surprise the results shown below are displayed.
I was definitely not expecting that (OK yeah I was because I created that script for this blog post, but just go with it). I then opened the file in notepad and it looked normal. So I edited it it notpad, saved it and the result was still the same. The first line was not being processed correctly, it seemed. As I was editing the file I noticed the encoding of the files were different. The encoding for the sample.cmd file was set to ANSI and for sample-vs.cmd UTF-8. See for yourself from the screen shots below.
So I switched the setting for the sample-vs.cmd file to ANSI, executed the script and all was good!
The past few days I was unable to Edit and Continue in Visual Studio 2010 Ultimate from my home machine, which is running Windows 7 64bit. I knew there were some issues with this on 64bit, I figured one of my projects was not targeting x86 so I just ignored it. Then today I was really getting annoyed at seeing the dialog.
So I started playing around with the configuration settings made sure that everything was in order and still no dice.
After that I performed a repair of Visual Studio, and then reset all the settings under Tools>Import and Export Settings and that still didn’t solve the problem. Then I remembered that I change my IntelliTrace settings the other day so I went to disable it and to my surprise I saw the dialog box shown below.
When I changed the IntelliTrace setting to collect call information I didn’t notice the warning stating “Edit and continue is disabled when collecting Call Information”! So I changed the setting and everything was good. Microsoft should add this to the Edit and Continue dialog box not available, I’ll ping a few people I know about that. FYI I created this question on Stackoverflow before I resolved it myself.
Recently I had the chance to work on a WPF app and it was behaving very strangely when I built and ran the application. For instance I tried to debug the application from Visual Studio and I received an error staging “Visual Studio cannot start debugging because the debug target ‘PATH HERE’ is missing. Please build the project and retry, or set the OutputPath and AssemblyName properties appropriately to point at the correct location for the target assembly.”
So I checked the properties, every thing looked good. I even built the project from the command line and it built fine. Anywayz at some point after that I was able to debug the app. But then I noticed that the UI was not getting updated with my changes. So I started digging a bit deeper. I noticed that when I ran a Clean from Visual Studio the output files for that particular project.
I then rebuilt the solution in Visual Studio. From the Output window I selected all the text,copied it, and then pasted that into an editor. After looking at the message
------ Skipped Clean: Project: R…ion: Debug Any CPU ------ Project not selected to build for this solution configuration ------ Skipped Clean: Project: R…Admin.Wpf, Configuration: Debug x86 ------ Project not selected to build for this solution configuration ------ Clean started: Project: Test…er, Configuration: Debug Any CPU ------ Build started 5/8/2010 1:10:02 AM.
I immediately knew that there was an incorrect value in the build configuration manager. You can find the configuration manager from the toolbar.
Here is what I found.
In this dialog there are two project set not to build for Debug builds; the DB project and the Wpf project. The DB project is OK to not build on Debug if there are a small % of DB changes versus code changes. If that is the case the devs can just build the DB project manually when the change it. In the case of the Wpf project, it should be set to build and the fact that it wasn’t was causing all the issues. So I checked the build check box, clicked the Close button and it was all good. You should also make sure that all other configuration as setup correctly. You don’t want to get bitten by this again.
If you are using Visual Studio 2010 then you may already be aware that Web Deployment Tool (aka MSDeploy) is integrated into Visual Studio. I’ve posted a few blog entries already about this tool. Two of the common questions that I get discussing this with people are
How do I exclude files from being placed in the package?
How do I add other files to the created package?
I will address these two questions here, first we look at the easier one, how to exclude files but we will go over a bit of background first.
Web Publishing Pipeline
With Visual Studio 2010 a new concept has been created which is known as the Web Publishing Pipeline. In a nutshell this is a process which will take your web application, build it and eventually create a package that you can use to deploy your application. This process is fully captured in MSBuild. With VS 2010 many targets and many tasks are shipped to support this process. Since its captured in MSBuild format, you can customize and extend to your hearts desire. So what we need to do is hook into this process to perform the customizations that we need. This process is captured in the following files.
The Microsoft.WebApplication.targets file is imported by the web applications projects file, then that file imports the Microsoft.Web.Publishing.targets file.
Excluding files from being packaged
If you open the project file of a web application created with VS 2010 towards the bottom of it you will find a line with.
BTW you can open the project file inside of VS. Right click on the project pick Unload Project. Then right click on the unloaded project and select Edit Project.
This statement will include all the targets and tasks that we need. Most of our customizations should be after that import, if you are not sure put if after! So if you have files to exclude there is an item name, ExcludeFromPackageFiles, that can be used to do so. For example let’s say that you have file named Sample.Debug.js which included in your web application but you want that file to be excluded from the created packages. You can place the snippet below after that import statement.
By declaring populating this item the files will automatically be excluded. Note the usage of the FromTarget metadata here. I will not get into that here, but you should know to always specify that.
Including extra files into the package
Including extra files into the package is a bit harder but still no bigee if you are comfortable with MSBuild, and if you are not then read this. In order to do this we need to hook into the part of the process that collects the files for packaging. The target we need to extend is called CopyAllFilesToSingleFolder. This target has a dependency property, PipelinePreDeployCopyAllFilesToOneFolderDependsOn, that we can tap into and inject our own target. So we will create a target named CustomCollectFiles and inject that into the process. We achieve this with the following (remember after the import statement).
This will add our target to the process, now we need to define the target itself. Let’s assume that you have a folder named Extra Files that sits 1 level above your web project. You want to include all of those files. Here is the CustomCollectFiles target and we discuss after that.
Here what I did was create the item _CustomFiles and in the Include attribute told it to pick up all the files in that folder and any folder underneath it. Then I use this item to populate the FilesForPackagingFromProject item. This is the item that MSDeploy actually uses to add extra files. Also notice that I declared the metadata DestinationRelativePath value. This will determine the relative path that it will be placed in the package. I used the statement Extra Files%(RecursiveDir)%(Filename)%(Extension) here. What that is saying is to place it in the same relative location in the package as it is under the Extra Files folder.
Admittedly this could be easier, but its not too bad, and its pretty flexible.
If you are using Visual Studio 2010 then you may already be familiar with the Web.config transformations that are now available. What you might not know is that you can use that same technology to transform config files outside of the build process. You will need Visual Studio 2010 installed on the machine where you perform these transformations. It is very easy to perform these transformation as well. Let’s say that we start with the app.config file shown below.
This MSBuild file uses the TransformXml task which is shipped with Visual Studio 2010. We specify the source file, transform file and the destination. Pretty straight forward.
In order to execute this I open a Visual Studio 2010 command prompt, browse to the directory containing both files, and enter the following command
msbuild trans.proj /t:Demo
Once you do this then you will find the file app.prod.config with the following contents.
A couple common questions that I’m frequently asked include
Should I use solution files for public builds
How can I replace a solution file with an MSBuild file
My answer for 1 is that I never use .sln files for any public build. I always create my own build file and use that. My take on solution files is that they are for developer convince for use in Visual Studio only. If you disagree, you are not alone.
Replacing a solution file with an MSBuild file is pretty easy, you just create an MSBuild file to build your projects. You will do that by using the MSBuild task. Now if you want to create a reusable .targets file (or set of .targets files) then this is a bit more involved, but makes it easier on you in the long haul.
I wrote about 20 pages on creating reusable .targets files in my book, but I'll get you started here with the basics here. I believe that the key to creating reusable build scripts (i.e. .targets files) is three elements:
Place behavior (i.e. targets) into separate files
Place data (i.e. properties and items, these are called .proj files) into their own files
Extensibility
.targets files should validate assumptions
The idea is that you want to place all of your targets into separate files and then these files will be imported by the files which will be driving the build process. These are the files which contain the data. Since you import the .targets files you get all the targets as if they had been defined inline. There will be a silent contract between the .proj and .targets files. This contract is defined in properties and items which both use. This is what needs to be validated.
The idea here is not new. This pattern is followed by .csproj (and other projects generated by Visual Studio). If you take a look your .csproj file you will not find a single target, just properties and items. Then towards the bottom of the file it imports Microsoft.csharp.targets (may differ depending on project type). This project file (along with others that it imports) contains all the targets which actually perform the build.
So it's layed out like this:
SharedBuild.targets
MyProduct.proj
Where MyProdcut.proj might look like:
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- This uses a .targets file to off load performing the build -->
<PropertyGroup>
<Configuration Condition=" '$(Configuration)'=='' ">Release</Configuration>
<OutputPath Condition=" '$(OutputPath)'=='' ">$(MSBuildProjectDirectory)\BuildArtifacts\bin\</OutputPath>
</PropertyGroup>
<ItemGroup>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary1\ClassLibrary1.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary2\ClassLibrary2.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\ClassLibrary3\ClassLibrary3.csproj"/>
<Projects Include="$(MSBuildProjectDirectory)\..\WindowsFormsApplication1\WindowsFormsApplication1.csproj"/>
</ItemGroup>
<Import Project="SharedBuild.targets"/>
</Project>
And SharedBuild.targets might look like:
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<!-- This represents a re-usable build file -->
<Target Name="SharedBuild_Validate">
<!-- See http://sedodream.com/2009/06/30/ElementsOfReusableMSBuildScriptsValidation.aspx for more info
about this validation pattern
-->
<ItemGroup>
<_RequiredProperties Include ="Configuration">
<Value>$(Configuration)</Value>
</_RequiredProperties>
<_RequiredProperties Include ="OutputPath">
<Value>$(OutputPath)</Value>
</_RequiredProperties>
<_RequiredItems Include="Projects">
<RequiredValue>%(Projects.Identity)</RequiredValue>
<RequiredFilePath>%(Projects.Identity)</RequiredFilePath>
</_RequiredItems>
</ItemGroup>
<!-- Raise an error if any value in _RequiredProperties is missing -->
<Error Condition="'%(_RequiredProperties.Value)'==''"
Text="Missing required property [%(_RequiredProperties.Identity)]"/>
<!-- Raise an error if any value in _RequiredItems is empty -->
<Error Condition="'%(_RequiredItems.RequiredValue)'==''"
Text="Missing required item value [%(_RequiredItems.Identity)]" />
<!-- Validate any file/directory that should exist -->
<Error Condition="'%(_RequiredItems.RequiredFilePath)' != '' and !Exists('%(_RequiredItems.RequiredFilePath)')"
Text="Unable to find expeceted path [%(_RequiredItems.RequiredFilePath)] on item [%(_RequiredItems.Identity)]" />
</Target>
<PropertyGroup>
<BuildDependsOn>
SharedBuild_Validate;
BeforeBuild;
CoreBuild;
AfterBuild;
</BuildDependsOn>
</PropertyGroup>
<Target Name="Build" DependsOnTargets="$(BuildDependsOn)"/>
<Target Name="BeforeBuild"/>
<Target Name="AfterBuild"/>
<Target Name="CoreBuild">
<!-- Make sure output folder exists -->
<PropertyGroup>
<_FullOutputPath>$(OutputPath)$(Configuration)\</_FullOutputPath>
</PropertyGroup>
<MakeDir Directories="$(_FullOutputPath)"/>
<MSBuild Projects="@(Projects)"
BuildInParallel="true"
Properties="OutputPath=$(_FullOutputPath)"/>
</Target>
</Project>
The important parts to notice are the extensibility points. Even though this is a very basic file, it has all the components of a reusable .targets file. You can customize it's behavior by passing in different properties and items to build. You can extend it's behavior by overriding a target (BeforeBuild, AfterBuild or even CoreBuild) and you can inject your own targets into the build with:
So this is the basics on how to create reusable build scripts which can help you easily create .proj files to replace your .sln files. I also recently answered. a related question Make a target run once at the Solution level in MSBuild.
I have seen a couple blog entries about executing MSTest unit tests from MSBuild. Most recently I saw the entry by Scott A. Lawrence. So I decided to share how I execute MSTest unit tests from MSBuild
I created a file named Build.Common.UnitTest.targets which contains all the behavior that will execute the test cases. This file can then be imported into whatever scripts that need to execute test cases. The entire file is shown below. We will discuss afterwards.
There are five important targets which are described below.
Name
Description
MSTestValidateSettings
This validates that the file was provided the needed data values to perform its task, to run the unit tests. For more info on this technique see my previous entry Elements of Reusable MSBuild Scripts: Validation.
MSTest
This is the target that you would execute to run the test cases. The target itself is empty but it sets up the chain of dependent targets.
CoreMSTest
This is the target which executes the test cases. This is preformed using the TestToolsTask.
BuildMSTestProjects
This target is responsible for building (i.e. compiling) the projects which contain the test cases. You don't have to call this it is called automagically.
CleanMSTest
This target will execute the Clean target for all the test projects defined.
If you take a look at the CoreBuildMSTestProjects target you can see that I am batching (Target batching to be specific) it on each defined configuration. This is achieved with the attribute Outputs="%(AllConfigurations.Configuration)". If you are not familiar with batching, see the links at the end of this post for more details, and you can always grab my book for even more detailed info J. Then inside that target I build each project by batching (Task batching) the MSBuild task on each project defined in the MSTestProjects item list.
Then inside the CoreMSTest target I execute the test cases. This target is batched for every value in the MSTestProjects item. As I'm writing this I have noticed that I've hard-coded the value for the configuration used in that target to be Debug with the statement
<PropertyGroup>
<_CurrentConfig>Debug</_CurrentConfig>
</PropertyGroup>
This shouldn't be hard coded, but passed in. I will leave it as is for now though. Then the TestToolsTask is invoked to execute the test cases.
Now that we have written the re-usable .targets file to execute the test cases we need to create a file which will "feed" it the necessary data values and let it do its magic. I created a sample solution, which you can download at the end of this post, which demonstrates its usage. The solution is named MSTestExample and you can see the files it contains in the screen shot below.
This file is pretty simple. It just creates some properties, and items and then just imports the Build.Common.UnitTest.targets file to do all the heavy lifting. You will probably notice that there are some properties defined that don't make sense here, like CodeAnalysisTreatWarningsAsErrors, this is because this was taken from a build script which does some other tasks. You can ignore those. To see what properties/items are required for the MSTest just look at the MSTestValidateSettings target. Also in the previous code snippet I showed how you could inject the MSTest target into the build process but it is commented out since there is no real build process in this case.
One thing about this approach that may not be ideal is that it will execute the test cases in one assembly at a time and if there is a failure it will not go on to the other assemblies. In my case this is OK because this is for local builds, public builds are executing test cases by Team Build, but this can be looked at.
Before I begin this post I must say that I do not have any inside information about the release of Visual Studio 2010 or .NET 4. The post is composed of 100 % speculation and based on information that has already been made publicly available.
I am pretty interested to know the release date for Visual Studio 2010, and .NET Framework 4.0, so I asked around but no one could tell me anything. So I decided to do some research as to the release dates for Visual Studio 2005 and Visual Studio 2008 based on the release of their Beta releases. I came up with the following table.
Yellow cells are 100% based just on the averages of the two previous versions
When Visual Studio 2005 came out there were huge changes. I think this is why the amount of time between Beta 1 and Beta 2, and between Beta 2 and RTM were so much longer then their corresponding periods in Visual Studio 2008. I think for the release of Visual Studio 2010 we are going to be somewhere between the two time periods. So maybe a decent indicator would simply be the average of the two. This is what I used to come up with the cells that are shaded in yellow. I'm sure this will be off but hopefully by weeks and not months!
If you are customizing your build process you may need to increase the verbosity that Visual Studio uses when building your projects. It is pretty easy to change this value, you just go to Tools->Options then find the Project and Solutions->Build and Run node. The dialog is shown here
The area where the verbosity setting lies is highlighted. I've got mine set to Detailed, which produces a lot of output. If you set it to Diagnostic, then you should be ready for a lot of output. Most like much more then you need!
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Monday, June 16, 2008 6:50:26 AM (GMT Daylight Time, UTC+01:00)
|
After some discussion at the MSBuild MSDN Forum I started investigating building the same project with different Compiler constants defined. I was actually pretty surprised by what I found. I created a simple Windows Form Application with a lone button one it. The image below show the simple app.
The parts to take note are the regions highlighted in yellow. So if the constant CONSTANT_ONE is defined the message results to CONSTANT_ONE and if CONSTANT_TWO is defined it will show the message CONSTANT_TWO.
Let's take a look at what's going on here. The BuildConstOne target passes the CONSTANT_ONE value as a property when invoking the MSBuild task, and builds to the binOne folder. The BuildConstTwo target passes the CONSTANT_TWO value and builds to binTwo. If you execute the target BuildAll, you would expect that the exe in binOne would show the message CONSTANT_ONE and the exe in the binTwo folder has CONSTANT_TWO message showing. What you actually get is that both actually show CONSTANT_ONE. This is because MSBuild doesn't entirely rebuild the project when building it for the second time in BuildConstTwo. The real reason behind this would be because both projects are building to the same BaseIntermediateOutputPath, obj\. There are two ways to of getting around this, one is to call Rebuild instead of Build, and the other is to specify the BaseIntermediateOutputPath as well. In the above I have demonstrated both approaches. The ReBuildAll calls Rebuild, and the BuildAllBetter specifies the BaseIntermediateOutputPath as well as OutputPath.
A question that comes up pretty frequently is "How can I debug an MSBuild task"? It's actually pretty simple. In this post I will describe how to easily an effectively debug MSBuild tasks that you are creating. In this example I will be demonstrating a task from my open source tasks at www.codeplex.com/Sedodream. The task is one that was contributed by Grant Holliday.
First in the project where your tasks are contained create a folder that will be used to contain sample MSBuid files that can be used to debug the tasks. This is also a good idea, because it will show people how to use your tasks. If you are don't want to mix samples & code in the same project then just make sure in your build you copy the files to the correct locations. In the sample project, which you can download at the bottom, the folder is named Samples. When you add MSBuild files to the folder make sure you set the file to be copied to the output folder as well. See the image below.
By setting this, the file will be copied to the output folder when the project is built. Since it will be in the output folder we can use a relative path to get to the assembly that contains the task. Take a look at the sample MSBuild file for this task.
The line highlighted contains the path to the assembly that contains the task. This path is relative to the location in the output folder, not the source folder. Also another thing to take note, is that we only write to the TaskLocation property if it is empty. This is useful because you can overwrite the location through command line parameters if necessary. But this shouldn't be needed for what we are trying to accomplish here.
After you create the sample, build the solution. And open a command prompt to verify that it works. Here is a sample of the result of this project file.
Once you've verified that that MSBuild file works then you can right click on the project that contains your task, and select properties. From there go to the debug tab. What we want to do is start the msbuild.exe executable on that sample project file. To do this fill in the full path to it in the Start External program text box, i.e. 'C:\WINDOWS\Microsoft.NET\Framework\v3.5\MSBuild.exe'. In the command line arguments you should pass the name of the project file followed by any msbuild parameters. Typically I will attach a file logger to the process. Finally you should set the working directory to the folder containing the sample. The result should look something like the image shown below.
Following this set a break point in your task, set the project as the startup project and hit F5! Another thing to take note of is that these properties are stored in the .user file so it shouldn't affect any other developers on your team.
Below is the link containing a simple solution that was used here.
Recently I was introduced to a tool that was to a tool that would analyze Javascript for syntactical errors as well as usage of bad practices. This tool is JSLint. Douglas Crockford from Yahoo! created, and maintains, JSLint. If you haven't heard of him, he is a well known Javascript expert.
The first line includes makes the tasks from my project available, and the second line includes the file that knows how to execute the JSLint tool. Following that I extend the build process to execute the RunJSLint target. If you are not familiar with this take a look at my article Inside MSBuild.
After you do this you may be prompted with a security warning from Visual Studio, you should pick 'Load Project Normally'. You can read more about how to disable it at http://msdn2.microsoft.com/en-us/library/ms228217.aspx. I chose to not have my installer set that registry flag. For the time being I think that users should make that decision themselves, although I would like to think I'm trustworthy J
After you do this and allow Visual Studio to load the project normally, if your create Javascript files that have errors, or JSLint doesn't like you will be warned in the error list as shown below.
With that being said keep in mind this is a V1 deal, so this may not work perfect. I am working to make sure that it works well, but it was kinda tricky to get this to work period!
As always I welcome your feedback and I will post more info about this later. Oh yeah by default from JSLint the GoodParts are enforced keep an eye on this blog, or send me a message, to see how to change that.
Previously I was thinking of writing some more material on MSBuild. Since then I have written about 60 pages of new material, and have actually decided to focus on some new topics. Because of this I will release a mulit part series that consists of this material that I have written. I think that I've come up with some pretty good content and would like to make it available. It still requires polishing and changes for this new format. So this is the first in this series of post.
These first posts will be from the MSBuild Quick Start chapter
In this post we will discuss the topics highlighted.
MSBuild Quick Start
Project file details
MSBuild Properties & Targets
Items
Simple Conditions
Default / Initial targets
Command line usage
Before / After build
Clean
MSBuild Quick Start
When you are learning a new subject it's exciting to get just dive right into it and get your hands dirty. In this section I will describe all the key elements you will need to know to get started using MSBuild. If you are already familiar with MSBuild feel free to skip this section, all of the material covered here will be covered in later sections as well.
The topics that we will cover in this section include; structure of an MSBuild file, Properties, Targets, Items and invoking MSBuild. With that being said, let's get started.
Project file details
An MSBuild file, typically called an MSBuild project file is really an XML file. These XML files are described by two XMLSchema Documents (XSD). These are located in the %WINDIR%\Microsoft.NET\Framework\v2+\MSBuild folder. Where v2+ is the version folder for .NET 2 and above. In this text I will assume you are using .NET 2.0 unless specified. In those folders you will find Microsoft.Build.Commontypes.xsd, this is the file that describes commonly found element in Visual Studio generated project files. Also this file includes the Microsoft.Build.Core.xsd document. This document describes all the fixed elements in an MSBuild project file. The simplest MSBuild file would contain the following:
This is the XML fragment that will identify this as an MSBuild file. Inside these tags is where all your content will be placed. Specifically we will be declaring Properties,Items, Targets and a few other items directly under the Project element. When you build you need to know what you are building, typically contained in Items and values that drive the build process, typically stored in Properties. We will now discuss how to declare properties, and we will cover Items later in this chapter.
MSBuild Properties & Targets
MSBuild properties are simply key value pairs, the key for the property is the name that you will use to refer to the property, and the value is, simply, its value. When you declare static properties they are contained in a PropertyGroup element directly under the Project element. Below is an example.
In the snippet above I have declared two properties; AppServer and WebServer with the values '\\sayedApp' and '\\sayedWeb' respectively. You can create as many PropertyGroups under the Project tag as you desire. The above fragment could have been defined as the one shown below.
If you create a sample project in Visual Studio, you will notice that there are many properties declared. These properties contain values that will be used throughout the build process. Below is a fragment from a Windows Application project that I created using Visual Studio.
You can see that things such as the output type, name of the assembly and many other values are defined in properties. Defining properties is great, but completely useless without being able to use them. So we will move on to Target declarations.
MSBuild has execution elements, those are Tasks and Targets. A Task is the smallest unit of work in and MSBuild file, and a Target is simply a sequential set of tasks. A Task must always be contained within target. Below is a sample that shows you the simplest MSBuild file which contains a target.
In this sample we have created a new Target named HelloWorld, but it actually doesn't perform any work at this point because it is empty. When MSBuild is installed you are given many tasks out of the box. You can find a list of these tasks at MSBuild Task Reference (http://msdn2.microsoft.com/en-us/library/7z253716.aspx). We will now use the Message task. The message task is used to send a message to the logger(s) that are listening the build process. In many cases this means that a message is sent to the console that is executing the build process. When you invoke a task in an MSBuild file you can pass it input parameters by inserting XML attributes with values. These attributes will vary from task to task depending on what inputs the task is expecting. From the documentation for the Message task (http://msdn2.microsoft.com/en-us/library/6yy0yx8d.aspx) you can see that is accepts a string parameter named Text. The snippet below shows you how to use the Message task to send the value 'Hello World' to the logger(s).
Now we will verify that this works as expected. To do this place the above text into a file named HelloWorld.proj. Now open a Visual Studio Command Prompt and go to the directory that contains the newly created file. The shortcut for this prompt is in the Visual Studio Tools folder under the Visual Studio folder in the Start menu. The command that you'll use is the msbuild.exe command. The basic usage for the command is
msbuild [INPUT_FILE] /t:[TARGETS_TO_EXECUTE]
So our command would be
msbuild HelloWorld.proj /t:HelloWorld
In the above we are saying, build HelloWorld.proj by executing the HelloWorld target. The result of this command is shown in the image below.
In the image above we can see that the HelloWorld target is executed and that a message 'Hello World!' was displayed on the console. Now let's see how we can use properties within other properties. Have a look at the simple project file shown below.
In the snippet above we can see that there are three properties that have been declared. On both the Configuration & Platform properties there is a Condition attribute. We will discuss these attributes later in this chapter. The remaining property, DropLocation, is defined using the values of the two previously declared items. Let's take a close look at that. The DropLocation Property has three components; a constant value and two values that are derived from the Configuration & Platform properties. When the MSBuild engine sees the $() notation it will replace that with the value of the specified properties. So the qualified DropLocation value would be: '\\sayedData\MSBuildExamples\Drops\Debug\AnyCPU\'. You can verify that by executing the PrepareFilesForDrop target from the Visual Studio Command Prompt. The reference for properties can be found at http://msdn2.microsoft.com/en-us/library/ms171458(VS.80).aspx.
When you use MSBuild there are a handful of properties that are available to you out of the box, that cannot be modified. These are known as Reserved properties. In the table below you will find all the Reserved properties.
Name
Description
Example
MSBuildProjectDirectory
Full path to the location of project file.
C:\MSBuild\MSBuild1\MSBuild1
MSBuildProjectFile
Filename of the project file, including extension.
MSBuild1.csproj
MSBuildProjectExtension
Extension of the project filename, including the initial dot.
.csproj
MSBuildProjectFullPath
Full path to the project file including the filename.
C:\MSBuild\MSBuild1\MSBuild1\MSBuild1.csproj
MSBuildProjectName
Name of the MSBuild project file excluding the file extension.
MSBuild1
MSBuildBinPath
Full path of the .NET Framework MSBuild bin directory.
%WINDIR%\Microsoft.NET\Framework\v2.0.50727
MSBuildProjectDefaultTargets
The value for the DefaultTargets attribute that is in the Project element.
Build
MSBuildExtensionsPath
Full path to the MSBuild folder located in the Program Files directory. This is a great place to keep commonly used items.
C:\Program Files\MSBuild
You would use these properties just as you would properties that you have declared in your own project file. To see an example of this you can look at Visual Studio generated project files. When you initially create a new C# project you will find the import statement <Import Project="$(MSBuildBinPath)\Microsoft.CSharp.targets" /> towards the bottom. This import declaration is using the MSBuildBinPath reserved property to resolve the full path to the Microsoft.CSharp.targets file. This is the file that drives the build process for C# projects. We will discuss its contents in more detail later in the book. Now that we have an idea how to use properties we will move on to Items.
Items
Building applications usually means dealing with many files. Because of this there is a specific construct that is used when dealing with files, and that is what's called Items. Items are usually file references, but they actually can be used for other purposes as well. If you create a project using Visual Studio you may notice that you see many PropertyGroup elements as well as ItemGroup elements. The ItemGroup elements contain all the defined items. When you define a Property you are declaring a key/value pair, but when you declare an Item you declare Items that can potentially have many elements inside of them. Let's see the syntax in action.
In this file, ItemsSimple.proj, there is an ItemGroup that has a sub element SoultionFile. The ItemGroup is the element type that all statically declared Items must be placed. The name of the sub elements are the names of the actual Items. In the above sample we have declared an Item names SolutionFile. The SolutionFile element has an attribute attached to it, the Include attribute. This attribute determines what actual entries are contained within the Item. This is because an Item can contain multiple entries. The Include attribute can contain a single value or a comma separated list of values. We will see this demonstrated later in this chapter.
In the target, PrintSolutionInfo, we can see the Item at use. Item evaluation syntax varies from Property syntax. To get the value of an item you will use the @() notation. The result of the build file, ItemsSimple.proj, is shown in the image below.
In this case the Item, SolutionFile, contains a single entry. Now let's see the result if there were multiple files included in the item. The snippet below shows the modified version of the ItemsSimple.proj file.
When you invoke the PrintCompileInfo target you will get the result 'Compile: Form1.cs;Form1.Designer.cs;Program.cs;Properties\AssemblyInfo.cs'. The Message tasks' Text parameter is a string value so the item is automatically converted, by MSBuild, into a string then passed to the task. This is done by concatenating all the entries separated by a ;. Later in the book I will demonstrate how you can customize this conversion process. Since the Include attribute can accept either one or many entries we could have actually defined the Compile Item as follows:
<ItemGroup>
<Compile Include="Form1.cs"/>
<Compile Include="Form1.Designer.cs"/>
<Compile Include="Program.cs"/>
<Compile Include="Properties\AssemblyInfo.cs"/>
</ItemGroup>
When a Property declaration appears after a previous declaration, this results in the value of the Property to be overridden. Items act differently from this; you cannot remove entries or overwrite values contained in an Item. You can only append to them. This is why both of the above statements are equivalent. Typically Items refer to existing files, if this is the case you can use wild cards to automatically include whatever files meet the conditions contained in the include statement. You can use three wildcard elements with MSBuild: ?, *, and **. You can use ? to replace a single character with any character. For example, the include declaration Include="c?r.cs" would include the files car.cs, cbr.cs, ccr.cs, and so on. The * element can replace any location with zero or more characters. To change the previous example, Include="c*r.cs" would include car.cs, caar.cs, cr.cs, colorer.cs, and so on. The ** notation tells MSBuild to search the directories recursively for the pattern. For example, Include="src\**\*.cs" would include all the files under the src directory with .cs extensions.
Another difference between Properties and Items is that Items can have metadata. Actually when you declare an Item each entry has a set of metadata associated with it out of the box. The table below outlines the well known metadata.
Name
Description
FullPath
Full path of this item including the filename.
RootDir
Root directory to which this item belongs.
Filename
Filename for this item, not including the extension.
Extension
File extension for this item.
RelativeDir
Path to this item relative to the current working directory.
Directory
Directory of the item without the root directory.
RecursiveDir
Used for items that were created using wildcards. This would be the directory that replaces the wildcard(s) statements that determine the directory.
Identity
Value for the item specified in the Include attribute.
ModifiedTime
Time this item was last modified.
CreatedTime
Time the item was created.
AccessedTime
Last time this item was accessed.
The basic usage for accessing metadata values is described by the template shown below.
@(ITEM_NAME->'%(METADATA_NAME)')
Where ITEM_NAME is the name of the item and METADATA_NAME is the name of the metadata that you would like to retrieve. Consider the following example. Assume that you have the following files in a directory; Class1.cs,Class2.cs,Class3.cs and Class4.cs. The following project file, MetadataExample01.proj, shows how to access the Compile Items' Fullpath metadata.
If you execute the PrintCompileInfo target you will see the following result.
From the output shown in the figure above we can see that the full path for all the Compile items was sent to the console. Accessing the metadata in this manner is given the term Item Transformation. We will discuss transformations in much more detail in a later chapter. Before we move on to discuss MSBuild Conditions take a look at a simple Windows Application project file that was generated by Visual Studio. You should see many things that you recognize.
Previously I blogged about an MSBuild Presentation that I was going to conduct at the JaxDug. I'm pleased to say that the presentation went well. I prepared a Powerpoint, but I decided to not use that and to discuss the various elements by example. I think that ended up being a lot more effective, and interesting than the Powerpoint would have been. You can find all the files for download below. The zip file contains all the files. The ppt is the Office 2003 version.
Earlier today a reader of my book emailed me the following question
I am trying to get a list of all the files being generated when MSBuild build acertain project (they should be all the files in .bin directory).
When I used the TargetOutputs in MSBuild task it gives me the main output only, which is the .exe or .dll. But any other files like dependency assemblies will not be listed in the TargetOutputs collection.
Any advice??
So it sounds like he is looking for a generic way of determining all the files that have been created (moved) during the build process. I’m assuming that he has a few projects and is using a build script to run the process, and then he wants to move the files to another location. If you have a few projects, then obviously the easiest way is to manually examine the output paths of each project and copy the file as necessary. This is not an ideal scenario, because if you add a project to your process you have to make sure to remember to perform the copy manually. Anywayz, this type of behavior would be useful in many different scenarios. Now let’s talk about how to achieve this.
I took another look at the Microsoft.Common.targets file and determined that there is not a straight forward method of performing this action. So, after thinking about this for a little while I think I have come up with a solution. It’s a little confusing, but actually quite simple.
The solution lies in what I’m calling “MSBuild Inheritance”, which I’ve blogged about previously in the entries at:
This is where you take an existing MSBuild file and without modifying it you add behavior and or data to it. Now let’s see how this idea can be used to add the desired behavior described above. For this sample I have created a sample Windows Forms app, as well as an external build script, buildWrapper.proj. This build file is shown below.
The purpose of this target is to define what targets it depends on and to create the outputs, by itself it doesn’t actually perform any action. Because of this we can override it in our buildWrapper.proj file. So the bottom half of the buildWrapper.proj file will conditionally import the project file and then conditionally re-define the Build target. Inside the new Build target we simply examine the output directory and create an item that contains all files that lie underneath it. In the image below you will see the result of this, specifically take a look at the region highlighted in red.
Now we can see that we have successfully achieved our goals.
Every time I describe this process I feel like it is still un-clear. If you have questions please let me know.
The MSBuild team is trying to prioritize their duties for the next release of MSBuild. Because of this they have asked for feedback on how their customers (us) would enhance MSBuild. They are doing this by distributing a virtual $100 across a possible 11 items. Their list actually ends in an item numbered 12, but there is no number 10. Here is a brief overview of their list of options.
1. Higher Preformance 2. VC Support 3. Support for other project types 4. Convert VS solution file to MSBuild format 5. Better dependency tracking 6. Distributed Build 7. Inline Tasks – ie you write a script right in your MSBuild file instead of a compilied .NET task 8. Scoping for items/properties – ie items/properties can exist in a defined scope 9. Extensible functions – ie being able to add items like the Exists function 11. MSBuild debugger 12. MSBuild visualization
Here is how I chose to distribute my $100.
4- $40 6- $10 7- $10 11 - $30 12 - $10
About #4: In my experience, you have 2 scenarios that are the correct way to create build scripts for products. Those are
1.Create a custom MSBuild file that acts like a solution file
2.Use a solution file to drive the build process
The only reason to go with 1 is because the solution file is not an actual MSBuild file, otherwise that wouldn’t even be an option. As for option 2, this is not ideal either because it is difficult to inject steps into how the solution build process works. All you can do is really extend what happens before/after the process.
As for the debugger, this is a tool that I have actually wanted to write myself for a long time, but never had the time. If there was a debugger for MSBuild, I think that more people would begin to adopt MSBuild as well as get a much better understanding of how it works.
I’ve created a new release of MSBuild tasks, at www.codeplex.com/sedodream. This release includes 4 .NET 3.5 tasks built on Visual Studio B2. Those are all Linq MSBuild tasks. They are:
Name
Description
CreateLinqCode
Creates a Linq code file from a database.
CreateLinqDbml
Creates a Linq DBML file from a database.
CreateLinqMap
Creates a Linq Map file from a database.
CreateDatabase
Creates a database from an Linq DataContext which is contained in an assembly.
The first three tasks simply wrap the SqlMetal command line utility. The last, CreateDatabse, will create a database from a Linq DataContext that is contained in an assembly. Right now there is little validation of the parameters passed, but I plan on adding additional validation. Also I will add more meaningful error messages, when invalid parameters are passed through.
Below you’ll find an example of the CreateLinqCode task.
Ok, I know there are many MSBuild loggers available so why create a new one? Because for a while I’ve been thinking that it would be cool to have a logger that would transform an MSBuild log into a log4net log. This is because obviously log4net has many appenders and tools that you can use out of the box. For example if you wanted to put your logs into a database, then attach the AdoNetAppender. Because of this, and because I couldn’t already find one, I wrote one. You can find it on my Sedodream MSBuild project. This logger is called SedoLogger. I figured this would be a good name, in the case that later on I wanted to abstract out the specific logging mechanism behind it. I have just created a new release that contains all the necessary files. I have posted some usage information about this logger on the SedoLogger WiKi page. I’ll be posting some more details about this logger here in the near future.
Someone contacted me regarding some issues he was having with respect to using the Web Deployment Project s and that was the inspiration for this post. There is an inherent problem with the way that Properties and Items are declared, created, overridden and used. There are a few problems those are:
·Items are append only, meaning that you can’t delete items or any reference contained within them.
We will now start by examining this issue. To simplify the issue I won’t use the Web Deployment Project files, or C# project files. But simpler files that let us focus on what is going on. Below is the file that represents the Microsoft.WebDeployment.targets file. This file is called Shared.proj.
Now let’s see the project file that represents the project that Visual Studio would create for you when you use the Web Deployments Template. My representative file is called Your.proj, it is shown below.
There are a couple of things to notice here, the first is that we are importing the Shared.proj file with the statement:
<ImportProject="Shared.proj"/>
The other is that we are overriding the SourceWebVirtualPath and the SourceWebPhysicalPath values, in the PreBuild event which is on the InitialTargets list. Because of this we know that the PreBuild target will be executed before any target that is not on the InitialTargets list.
Now going back to the Shared.proj file, we can see
So from this we are creating a new property _AspNetCompilerVirtualPath based on the original SourceWebVirtualPath, and and item _SourceWebPathItem based on SourceWebPhysicalPath. These new properties and items are used throughout the Microsoft.WebDeployment.targetsfile, and is a common practice amongst other .targets files. You’ll find something similar in the Microsoft.Commons.targets files. Now is where the problem arises, these new properties and items that have been declared in the PropertyGroup & ItemGroup, so they will be evaluated and assigned before the PreBuild target executes. Because of this you don’t really have a chance to actually override these values using any logic. The only (until you read further) way you can override them is to use static elements like PropertyGroupand ItemGroup.
You can see this is the case from the output of the PrintValues target below.
Target PreBuild: Changing path to be 'OtherPath' instead of 'WebPathItems'
So how can we fix this? We need to be able to dynamically create properties & items and have them override static items? We can create a wrapper MSBuild file that does the logic to determine the correct values and the use the MSBuild Task. Have a look below at the file Fix.proj.
Take a look at the highlighted portion, it simply calls MSBuild on the Your.proj fle. Here is the output when executing this file.
Target PreBuild: Changing path to be 'OtherPath' instead of 'WebPathItems' SourceWebVirtualPath: OtherPath SourceWebPhysicalPath: OtherPath Target Build:
__________________________________________________ Project "C:\Data\Community\msbuild\WebDeploymentIssues\ASPNETix.proj" is building "C:\Data\Community\msbuild\WebDeploymentIssu\Sample\Your.proj" (PrintValues target(s)):
Target PreBuild: Changing path to be 'OtherPath' instead of 'WebPathItems' Target PrintValues: SourceWebVirtualPath: OtherPath SourceWebPhysicalPath: OtherPath _AspNetCompilerVirtualPath: OtherPath _SourceWebPathItem: OtherPath
Build succeeded.
So you can see that all the desired values have been overridden. But having 2 files is really annoying, and in some cases you don’t really have this option. So because of this we need to find a solution that will modify the original file only. And here is that file, it is shown below its called YourFix.proj.
In this approach we need to modify the behavior of the PrintValues target so we override it and have it call itself with the correct values. The MSBuild ProjectFullPath is an MSBuild Reserved Property, and it will return the full path of the current project file. The trick here is the conditional override of this target based on the DoingFix value. So we can see that the DoingFix property will default to true, so the PrintValues target will be overridden. But when the file is called back into with DoingFix=false it will not be overridden, and the PrintValues in Shared.proj will be used. In your use this target will most likely be Build. The drawback of the approach is that you’d have to overwrite each target that you are interested in. The other drawback is that it is confusing if you are not paying close attention to what this does. I posted something previously that used a similar technique read it if this interests you http://www.sedodream.com/PermaLink,guid,9c235811-3078-45e7-b122-6a239fb33fc0.aspx.
Here is an outline of the files I have attached:
Shared.proj=> Represents a shared file (ie Miscrosoft.WebDeployments.targets)
Your.proj=> Represents your extension of the shared.proj file (ie SiteDeployment.wdproj)
Ok, here is the challenge you have several project files that comprise a set of products. Now you want list every reference that each project file has. How do you do it? I came up with a pretty nifty way of doing it. If you look a project file for a managed project (I’ll be using C# projects for this example) you’ll see that you can see all the references listed in an Item named Reference. So there’s got to be a way to extract out that information for each project. I was thinking that it would be nice if each project had a target that would simply output that item. But that would involve changing each project to add the target to perform that. Which obviously is not an option. But what we can do is achieve the exact same thing, by using the Import element. Take a look at the simple targets file below:
What this will do is import an MSBuild project using the Import statement, this file is driven off of a property file. After that it will merge a new target GetReferences with the project declared in ProjectFile. Now we simply need a way to specify that ProjectFile. To achieve this we will actually change the above statements to include the condition Condition="$(ProjectFile)!=''". And we add the following statements to find all project files, enumerate references and write a file containing all of them.
If we execute the FindReferences target on this project file it will batch over each project file. It will then execute the same project file with the property ProjectFile defined, and execute the GetReferences target. So what is really happening is that we are effectively extending the existing MSBuild project file and injecting a new target to an existingproject file. Pretty nifty.
Here is the entire project file so you can see the end result, also you can download it below.
Just wanted to let everyone know that my Sedodream MSBuild Project now has an MSI that can be downloaded. You can find that latest downloads at http://www.codeplex.com/Release/ProjectReleases.aspx?ProjectName=Sedodream. The installer will place all MSBuild tasks in the %Program Files%\MSBuild\Sedodream directory. In your MSBuild project files you can refer to this location as $(MSBuildExtensionsPath)\Sedodream\.
Here is a list of what is currently included in the project
Tasks Included
Name
Description
AddMetadata
Allows you to add metadata to an item
FindUnder
Finds and returns all directories (empty or not) under a specified path
GetDate
Returns a string representation of the date in a specified format
GetRegKey
Returns the value of a registry key
Move
Task to move a file (or set of files)
NUnitTask
Task to run NUnit test cases as a part of the build process
TempFile
Returns the path to a new temporary file
ReplaceInFile
Can be used to replace specified text in a file.
GetExternalIp
Can be used to determine the external IP address of the executing machine
GetInternalIp
Can be used to determine all the Internal IP addresses of the executing machine
Loggers Included
Name
Description
EmailLogger
An MSBuild logger capable of emailing the resulting log
There were a couple of posts in the MSBuild forum recently related to customizing the build process. The specific thread is, Forcing a Clean build. The question boils down to, "How do you force a clean each time I build?" The answer lies in adding the Clean target to the build targets dependency list. You can read details on how this works in my MSDN article Inside MSBuild. One of the participants asked me to post a complete solution of how to do this. So that response is this entry. I created a sample Windows Application that has the build process customized. I called this BeforeBuildEx, you can download all the files at the end of this entry. I modified the WindowsApplication1.csproj file and added the following lines:
<PropertyGroup>
<BuildDependsOn>
Clean;
MyBeforeBuild;
$(BuildDependsOn);
MyAfterBuild
</BuildDependsOn>
</PropertyGroup>
<TargetName="MyBeforeBuild">
<MessageText="This is before the build."Importance="high"/>
</Target>
<TargetName="MyAfterBuild">
<MessageText="This is after the build."Importance="high"/>
</Target>
The list of targets that must be executed for a build is contained in the BuildDependsOn list. So if you change that list, you change to build process. If you want to add a target before the build process, add that target to the begining of the list. In the above statement, I'm actually extending the previous list by adding my targets. I add 2 targets before and 1 after the already existing definiton for BuildDependsOn. Much more detail about this in my article. The snippet above was inserted after the statement
This is very important, otherwise your customizations may be overwritten. Now if you build this you will see that the Clean target is executed when the application is built. If you want to see this in Visual Studio you have to increase the verbosity of the MSBuild output that is shown on the Output window. Do this by: Tools->Options->Projects and Solutions->Build and Run and set the value for MSBuild project build output verbosity to atleast Normal. The link to the zip file is below. In some cases this could cause some problems for Visual Studio.
A twist on the previous entry. Here’s a very similar, but different scenario. You have a set of files, that you want to push to a set of folders. But you have to “discover” these folders. In this scenario there are really 2 different situations you can be in.
1.Each folder contains a file that you are able to identify by file name
2.Each folder contains no files, but you are able to identify the folder by its name
For 2) you’ll have to rely on a custom task to get locate these folders. This is because if the folder is empty then you are not able to put these in an Item the normal way. I’ve previously written a task FindUnder that you can use for this purpose. You can find this task, and a handful of others at http://www.codeplex.com/Wiki/View.aspx?ProjectName=Sedodream. Let’s go over scenario 2 because I think it’s a little more straight forward.
Here is the directory structure that you have to work with:
│AssemblyInfo.cs
│DeploySample.proj
│Other1.cs
│Other2.cs
├───Deploy
│├───five.control
│├───four.control
│├───one.control
│├───six.control
│├───six.something
│├───three.control
│├───three.something
│└───two.control
In this example the folders that you want to copy into end with .control, and the files you want to copy include AssemblyInfo.cs, Other1.cs and Other2.cs. In this situation we want to find all folders under Deploy that end with control. So our search pattern will be *control under the Deploy folder. Here’s how we find those folders using the FindUnder task, and copy the files into them.
Let’s talk about the Copy task above. This shows how we can copy a set of folders to a set of different folders. This is possible through batching which was discussed in more detail in the previous post. As you use batching keep in mind that batching isn’t limited to custom meta-data. But can also be used with Well-known item metadata. That is what we are doing in this example.
Now let’s see how can we handle situation 1). To review, this is the situation that we have files in a set of directories that we can identify. In this example let’s say that file is named app.config. So we want to find all folders under a given location that have a file named app.config inside of it. So here is your directory structure
│AssemblyInfo.cs
│DeploySample.proj
│Other1.cs
│Other2.cs
└───Deploy2
├───five.control
│app.config
├───four.control
│app.config
├───one.control
│app.config
├───six.control
│app.config
├───six.something
├───three.control
│app.config
├───three.something
└───two.control
app.config
So the result should actually be the same as the previous example. This is how to achieve that.
<PropertyGroup>
<DeployRoot2>Deploy2</DeployRoot2>
</PropertyGroup>
<ItemGroup>
<KeyFilesInclude=".\$(Deploy2)\**\app.config"/>
</ItemGroup>
<TargetName="Deploy2">
<MessageText="Folders to deploy to:%0d%0a@(KeyFiles->'%(RecursiveDir)','%0d%0a')"/>
So the in this example I’m using the KeyFiles item to determine which folders to copy all the files to. From the Deploy2 target you can see that I’m using batching to copy all the FilesToCopy into each of the locations determined by the KeyFiles item. The directory that contains each KeyFile is determined by the RecursiveDir well-known meta-data value.
Today someone was telling me about a co-worker who was having issues with MSBuild. He told me that he was trying to copy a set of files to a set of different servers. But the issue was that he didn’t know how to achieve this without performing multiple Copy task invocations. I told him that he could achieve this using MSBuild Batching. Batching is a process of performing a task (or target) on a set of items (batches) at a time. A batch can also include a single item. So in this scenario we need to perform the copy one time for each server that he wanted to deploy to. I’ve created a simple msbuild file which demonstrates this in two different ways. The first way uses task batching, which can bee seen in the Test target. And the other uses Target batching which can be seen in the DoItCore target. I've also created a clean target, which has nothing to do with batching.
Batching is an advanced topic of MSBuild, and is defintely neglected. I have to admit I’m guilty of not writing about it enough myself. There are some good batching resources, they are listed below.
Previously you could only get the MSDN Library if you were an MSDN Subscriber. I think its great that Microsoft made the MSDN Library free to download. It's a great resource and contains all kinds of goodies for .NET developers.
Sayed Ibrahim Hashimi
Visual Studio
Wednesday, August 9, 2006 5:32:45 AM (GMT Daylight Time, UTC+01:00)
|
I've created a project on CodePlex to hold all of the MSBuild tasks/loggers and what not from my book and my MSDN article Inside MSBuild. The project home page is at Sedodream MSBuild CodePlex project. The idea is that I'll keep all of the MSBuild related material that I discuss here there as well. If you have content that you'd like to post there thats great too. Here is the current list of what is available there:
Tasks Included
Name
Description
AddMetadata
Allows you to add metadata to an item
FindUnder
Finds and returns all directories (empty or not) under a specified path
GetDate
Returns a string representation of the date in a specified format
GetRegKey
Returns the value of a registry key
Move
Task to move a file (or set of files)
NUnitTask
Task to run NUnit test cases as a part of the build process
TempFile
Returns the path to a new temporary file
Loggers Included
Name
Description
EmailLogger
An MSBuild logger capable of emailing the resulting log
SimpleFileLogger
A simple MSBuild file based logger
XmlLogger
An MSBuild logger which creates an Xml file
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Tuesday, July 18, 2006 5:47:05 AM (GMT Daylight Time, UTC+01:00)
|
A few days ago I received an email about an MSBuild visual editor! This was something that I was interested in doing a while ago, but never got around to doing it. This tool is called MSBuild Sidekick, and you can download it for free from http://www.attrice.info/msbuild/index.htm. I have downloaded it and played around with it. I think it’s a good tool for people who are getting started with MSBuild, but for those who use MSBuild on a regular basis I don’t think it can replace the productivity that is achieved with Intellisense. They have an interesting way of representing the build file visually, and after some time if the right extensions are added it may be a tool that I’d use over Visual Studio to edit my MSBuild files. One of the challenges of representing a build file is that you have to be aware of where your new declarations are being placed. For instance if you have two properties defined, lets say Property1 and Property2. If Property2 depends on Property1 then you have to make sure that it is declared after Property1. As far as I could tell there was no way to achieve this without viewing the source of the project file through the tool.
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Thursday, June 15, 2006 6:14:45 AM (GMT Daylight Time, UTC+01:00)
|
In previous versions of the Visual Studio the build customizations were basically limited to a PreBuildEvent and a PostBuildEvent. Now we have MSBuild which allows for many very specific and complex build customizations. But Microsoft obvioiusly must continue to support the Pre/Post build event. It is something that already exists, many people are familiar with it, and its very simple. Let’s see how this is implemented within MSBuild. If you add a Pre or Post build event from Visual Studio, then examine yore project file you’ll find something like the following declared.
<PropertyGroup>
<PostBuildEvent>echo 'This is the post build event!'</PostBuildEvent>
</PropertyGroup>
So these events are actually simple properties that will be executed when the time is right. What does this mean for you? Now lets talk about what’s funny about this. When MSBuild loads your project file it will load all of the properties contained in PropertyGroup elements, followed by that your items will be evaluated. What this means for the Pre/Post build event is that you are able to access properties that have been declared in PropertyGroup element before the Pre/Post build event declaration. After this evaluation is made the value for the Pre/Post build event will not change! What this equates to is that you are able to use properties such as OutDir or AssemblyName, but you’ll never be able to access properties that are created from tasks contained in other targets! If you need to access properties that have been created by other targets then you’ll have to extend your build process outside of the Pre/Post build event. This is outlined in my MSDN article Inside MSBuild in the Extending the Build Process section.
If you look around the net
you'll find a few different Xml loggers for MSBuild. When I was writing my book
I decided to write my own Xml Logger for a few different reasons. The source
for the Xml logger that I wrote is available at apress.com.
I'm currently in the process of trying to get some shared space for me to place
my tasks and loggers that I've written for everyone to have a look at. Once I
get that squared away I'll make sure to write an entry about it. In the mean
time let's talk about my Xml logger. What is different about my Xml logger
versus some others that you may find? When I was looking around for one, I
noticed that the ones that I found didn't support building solution files very
well. So that was a deficiency that I wasn't willing to sacrifice with. Besides
that, I noticed that none of them had an append options as most file based
loggers do. So I added that to my Xml logger. I think this is a good option if
you are performing a nightly ( or continuous ) build. That way you have one
place to go to each time you want to see how your build progressed. However
this xml file can get quite large if your build is spanning many projects.
At the bottom of this blog you can download the dll for this logger (Contained
in a zip file). The source will be made available soon. There are 4 logger
arguments:
Name
Description
Append
True/false If true then if a
log file exists then it will be append to, otherwise any existing log file
will be replaced.
Logfile
Name of the log file to write
to
Showsummary
True/false if true then a
summary will be contained in the xml file for errors and warnings.
Ok, there's several posts about whats good/bad about both of these languages, this isn't one of those. At the end of the day it doesn't matter which technology is the best, but which ones are going to employ you. About a year ago, I think there were many more Java jobs here in Jacksonville (FL) then there were C# (.NET) jobs. I've noticed that there has been an increasing need for C# developers here, can't really say about the Java positions though. Recently I've been playing around with the blog search engine technorati.com, they have some really cool features. If you've never been there you should give it a shot. One of the features is to chart the number of blog entries over a period of time, based on keywords. To view the graph of MSBuild related blogs click on the link below
I decided to compare the results of a search of "C#" and "Java", and I was really surprised to see the results. Below you can see the images that I saw. Note: These images are static and not updating.
From the technorati.com site it stated that there was 668,373 Java related posts and 17,100,527 C# posts! That is an incredible difference. There could be a few reasons for the difference:
Java people arn't blogging as much
Java people arn't registering with technorati.com as much as the C# folk
There is something wrong with the technorati.com processor
Since this is a simple keyword search I think we can safely assume that (3) is not the culprit. I think its really a combinition of (1) and (2). Another aspect there are many Java related blog entries that don't actually have the word Java in it. But I'm sure the same goes for C#. Previously when I was doing mostly Java work, I thought one of the cool things about Java was the community effort. With the open-source side and all. I thought that C# (.NET) would lack this for sure. But now that I know better, I know that it certainly is not like that. I think the community effort for C# (.NET) is just as strong, if not stronger, then the Java side.
I'm not sure what the significance these number have, but it certainly does make be happy to be in the C# camp. :)
Frequently I hear people asking "How can we use Visual Studio 2005 for 1.1 development". If you are writing managed code (C#,J#, or VB.NET) then you are in luck. There is a toolkit that was released from the guys at Microsoft that can do this for you. Its called MSBee (MSBuild Everett Edition). You can download this toolkit at Codeplex. The creator of MSBee blogs about it at: http://blogs.msdn.com/clichten/default.aspx
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Saturday, June 3, 2006 7:38:13 AM (GMT Daylight Time, UTC+01:00)
|
I received this email from a reader of my MSBuild MSDN Article, here is his question:
I have multiple projects in multiple directories on my machine, I need to compile all of those projects and copy the resulting DLLs into my application folder which id different from the projects. Can i do all this in one MSBuild file? Most the MSBuild file samples i find work only on a single project and assume that the build file is in the same directory as the .CS files.
To answer this question you can create an MSBuild project file which uses the MSBuild task to build other project files. By using this you can extract out the files that were built during the process and place them into an item to be copied to the location of your choice. To demonstrate this I have created the following directory structure:
├BuildSeveralExample
│├───App1
││└───WindowsApplication1
││└───Properties
│├───App2
││├───ClassLibrary1
│││└───Properties
││└───ConsoleApplication1
││└───Properties
│└───Deploy
The App1 and App2 folders each represent a product that is to be built. App1 contains a C# Win Forms project and App2 contains a C# Class Library and a C# Console App. The Deploy folder is the location that we would like to copy all of the output files to. In the BuildSeveralExample folder I have created a projects file, BuildAll.proj, which can be used to fulfill this need.
The first thing we need to do is to have a way to locate all of the project files, in my solution I created an Item and included all of the project files as such: <ItemGroup>
I include all project files in the current directory or any subdirectory, with the exception of the current project file. The MSBuildProjectFile is an MSBuild Reserved property, for a complete list see: http://msdn2.microsoft.com/en-us/library/ms164309.aspx.
In your situation you may need to specify what files to include and exclude, to do this you can simply enter their locations separated with a semi-colon. Now that we have all the project files we’d like to build we need to actually build them. Do this by:
This is using the MSBuild Task to each of the project files. Also note the use of the Output element here, the MSBuild Task has an output of TargetOutputs which contains a list of files which were produced by the project files built. These files are placed in the OutputFiles item. Since we didn’t specify a target to be executed on each project file, the default target will be executed. This is how Visual Studio builds your project files as well. By using the MSBuild task we can also specify properties to be passed to the project file(s) in the Properties attribute. If you need to pass different properties for different project files, then you may have to call the MSBuild Task more than once.
Now all we have to do is copy these files over to another location. This is achieved like:
The location to copy the files is specified in the DestFolder property shown above.
As well as performing these actions, we’d also like to create a fresh build when this process is invoked. To do this we can either call the Rebuild target on the project files or call Clean before we build them. I prefer the second approach, but both are equally good. It also serves for a better demonstration for me to implement the second approach.
To clean the projects all we have to do is call the Clean target on each of the project files. I have written about this in more detail in my previous entry Clean many projects and Extending the Clean. As well as that we have to remove any files that have been created by a previous invocation of this process, have a look at my clean target:
<!--
This target can be used to clean all of the projects before you build them.
It will also delete any dll & exe files located in the the Deploy folder. This could be accomplished
much better than this, but this is quick and easy.
-->
<TargetName="CleanAll">
<!-- Delete any files this process may have created from a previous execution -->
I took the Visual Studio approach and defined targets to perform individual steps, created one target to call them in the correct order by using a depends list derived from a property. This is a good approach because this depends on list can be modified outside of this project file to inject other required steps. For more detailed info on that see my other entry Extending the clean.
To see this in action you have to execute the BuildAll target on the BuildAll.proj file. I’ve made the project file available as well as my sample applications that go along with it. The BuildSeveralExample.zip file contains all the necessary files.
This is the beginning of a new series of blogs that I plan on writing. Its called ‘MSBuild Fun’, the idea is that I’ll discuss some examples of how you might use MSBuild, outside of building applications. MSBuild is a tool that certainly is geared toward building applications, but is not limited to that. Knowledge of MSBuild can help you with some small things that come up quite often.
In this example we will consider a relatively simple task, copying a set of files from one location to another. Let’s say that you would like to locate all image files contained in a particular location and copy those files to another. There are a few options; you can use the Windows Explorer to search for all of the image files, and copy-paste them to the other location. There are many problems associated with this approach:
If there are files with the same name in a different folder then you will only be able to get 1 copy of it
If there is an error during the course of copying the files it will stop all together
This doesn’t preserve directory hierarchy
You can only include files to be copied, not exclude them
Another option is to use the xcopy command from the command prompt. This is a good method to employ to solve the problems associated wit the previous approach. Using this approach we can continue on error, and preserve the directory hierarchy.We can also exclude files from being copied. If you are to perform this task many times that you can simply insert the call into a .bat file to preserve the command. The approach we will use here is to use MSBuild Copy task to copy the files from the source to the destination. It solves all the problems associated with the first approach, and throws in a few bonuses. Those are:
Syntax for excluding files is very powerful, much better than that for xcopy
You can transform the name of the file as you copy it
You can skip files that already exist, and haven’t changed, in the destination location
In one MSBuild file you could house various different copies. With the batch file approach you are basically limited to one pre file.
From a performance perspective the xcopy and MSBuild approach are pretty much equivalent, so that is not a factor. Now lets have a look at this in action.
This simple MSBuild project file will copy all Word Documents and text files that don’t contain trash in the filename to another location maintaining the directory hierarchy. As you can see here, in the FilesToCopy item declaration, I’m using the include attribute to include a list of files to be copied, and the exclude attribute to exclude a list of files from being copied. You can place complex wildcard expressions in these attributes to select only the files you are truly interested in.
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Wednesday, May 3, 2006 6:02:18 AM (GMT Daylight Time, UTC+01:00)
|
In the June 06 issue of the MSDN magazine you'll find an article that I wrote titled "Inside MSBuild: Compile Apps Your Way With Custom Tasks For The Microsoft Build Engine". As far as I know this will be the first article in print for the release version of MSBuild. Before I wrote this article I noticed a need, that was there was no one good resource for people to go to to learn all the key things they need to really get going with MSBuild. I think this article really does meet that need decently well. The only large topic that wasn't covered in the article is writing custom loggers. There are a few reasons for this; limited number of words, most users won't be interested in writing custom loggers and there is a lot of material about writing custom loggers available. If you are interested in writing a custom logger, I've got examples on this blog and here are a few other places you can visit: http://msdn2.microsoft.com/en-us/library/ms171471.aspx http://blogs.msdn.com/msbuild/archive/2005/10/10/479223.aspx
Along with these in my book I present a custom XML logger. If you read the article I welcom your feedback.
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Friday, April 21, 2006 3:13:06 PM (GMT Daylight Time, UTC+01:00)
|
If you are using Team Foundation Server as your SCM tool, I hope you're taking advantage of using the Team Build tool to create re-producable public builds for your projects. Very cool, and very useful, especially for those Enterprise applications. As you use Team Build you'll need to make changes to how your build process is executed. You can make the customizations in the TFSBuild.proj file. This is the file that will drive your team build. In this file there are many properties declared that are generated by the New Build Type Creation Wizard. These properties are contained in the first PropertyGroup element in that file. Some of these properties include; Description, DropLocation,BuildDirectoryPath,... which are used internally by Team Build to setup the build. Of course Team Build uses MSBuild to actually execute the build, but some things need to happen before/after MSBuild gets involved. For instance creating the Build report, which require those values. In the ideal world you'd expect for those values to be gathered using the MSBuild Object Model, but this is not the case. Team Build is extracting those values as they are declared. There has been a few posts about this particular problem on the MSBuild MSDN Forum. Those posts are at:
In the post Property Evaluation Order (beta 3) I provide a workaround for most situations, but its not always perfect. Here it is for you to see:
It seems like Team
build is not using the msbuild object model to get those evaluated
properties, but instead are simply extracting the text value from those
property declarations.
I tried to overwirte it in different places in the project file as
well, with no luck. It always took the first declaration. I'm guessing
that the property is not able to be overwritten because it is passed in
as a command line parameter to msbuild.exe by team build. But there is
a workaround that solves some of the issues. In your TFSBuild.proj
file, modify the DropLocation to one that doesn't use any properties
and insert the following towards the bottom:
This prepends the FixDropLocation to the DropBuildDependsOn list of targets, so it will be executed before the DropBuild target.
This cause your project drop files to be dropped in the location tat
you want. This is able to overwrite the DropLocation property becuase
we are using the CreateProperty task to overwrite this instead of the normal property declaration.
This is not however a perfect solution, because your build log will always be located in the DropLocation
that was originally declared (the one w/o the property embedded within
it). And in the Team build type history the drop location will also
have this location as well. But it does place the files where you want
them :)
Unfortunately it looks like the DropLocation that is given to TFS is
happening outside the scope of MSBuild so I'm not sure you have a lot
of options. You can't even pass this as a command line parameter to the
tfsbuild.exe utility either. Let me know if this is not clear.
In case you haven't already heard a service pack will be released for both VS 2003 & VS 2005 this year! SP1 for VS 2003 is scheduled to ship during the second quarter, and SP1 for VS 2005 in the third. You can read more about this at: http://msdn.microsoft.com/vstudio/support/servicing/. I think its kind of interesting that VS2003 didn't have a service pack for 3 years, yet 2005 is having one not even 1 year later. Perhaps because of all the new features that were shipped with this version, however VS 2003 shipped with many new features as well. Since MSBuild is technically a part of the .NET Redistributable, I'm not sure if the service packs will impact anything related to MSBuild. If I find out more I'll be sure to post more.
Sayed Ibrahim Hashimi
Visual Studio
Friday, March 31, 2006 7:07:45 AM (GMT Daylight Time, UTC+01:00)
|
Previoiusly I wrote about the Microsoft SDC Tasks library, there is another great resource for MSBuild tasks. This is the MSBuild Tigris Tasks project. This is an open source project, so you can submit your tasks to this library as well. Tigris.org is known for their Source Control Management (SCM) tool, Subversion. I've used Subversion previously and was pleased with it, but my favorite is Team Foundation Server :) If you're using Subversion as your source control you'll be glad to find many task for accessing your repository. From the project's site here is a list of the MSBuild tasks that are either completed or currently under development.
Task
Description
AppPoolController*
Allows control for an app pool on a local or remote machine with IIS installed.
AppPoolCreate*
Creates a new application pool on a local or remote machine.
AppPoolDelete*
Deletes an existing application pool on a local or remote machine.
AssemblyInfo
Generates an AssemblyInfo file using the attributes given.
Attrib*
Changes the attributes of files and/or directories
FileUpdate*
Replace text in file(s) using a Regular Expression.
FtpUpload
Uploads a file using File Transfer Protocol (FTP).
FxCop*
Uses FxCop to analyze managed code.
Mail
Sends an email message.
Math.Add
Add numbers.
Math.Divide
Divide numbers.
Math.Multiple
Multiple numbers.
Math.Subtract
Subtract numbers.
Move*
Moves files on the filesystem to a new location.
NDoc
Runs NDoc to create documentation.
NUnit
Runs tests using the NUnit.
RegistryRead
Reads a value from the Registry.
RegistryWrite
Writes a value to the Registry.
Script*
Executes code contained within the task.
ServiceController*
Task that can control a Windows service.
ServiceQuery*
Task that can determine the status of a service.
Sleep*
A task for sleeping for a specified period of time.
SqlExecute*
Executes a SQL command
SvnCheckout
Checkout files from Subversion
SvnClient
Subversion Client
SvnCommit
Commit files to Subversion
SvnExport
Export files from Subversion
SvnInfo*
Get Subversion information for a file or directory.
SvnUpdate
Update files from Subversion
SvnVersion
Get Subversion revision number of a local copy
TaskSchema*
Generates a XSD schema of the MSBuild tasks in an assembly.
Time*
Gets the current date and time.
Unzip
Unzip a file to a target directory.
Version
Increments a four-part version number stored in a text file
VssAdd*
Adds files to a Visual SourceSafe database.
VssCheckin*
Checks in files to a Visual SourceSafe database.
VssCheckout*
Checks out files from a Visual SourceSafe database.
VssClean*
Removes VSS binding information and status files from a solution tree.
VssDiff*
Generates a diff between two versions of an item in a Visual SourceSafe database.
VssGet*
Gets the latest version of a file or project from a Visual SourceSafe database.
VssHistory*
Generates an XML file containing the history of an item in a VSS database.
VssLabel*
Labels an item in a Visual SourceSafe database.
VssUndoCheckout*
Cancels a checkout of an item from a Visual SourceSafe database.
WebDirectoryCreate*
Creates a new web directory on a local or remote machine.
WebDirectoryDelete*
Deletes a web directory on a local or remote machine
WebDownload*
Downloads a resource with the specified URI to a local file.
XmlRead
Reads a value from a XML document using a XPath.
XmlWrite
Updates a XML document using a XPath.
Xslt*
Merge and transform a set of xml files.
Zip
Create a zip file with the files specified.
* Items not complete or have not been released.
Sayed Ibrahim Hashimi
msbuild | Visual Studio
Thursday, March 30, 2006 4:21:48 AM (GMT Daylight Time, UTC+01:00)
|
One of the Microsoft teams based in the UK has created a library of MSBuild tasks that is available for you to download at: http://www.codeplex.com/sdctasks You can download binary versions as well as all the source code for these tasks. There is almost 170 tasks that they have made available! Below is a list of them all if you are interested in what they are.
ActiveDirectory.Group.AddUser
SourceDepot.ReverseIntegrate
ActiveDirectory.Group.Create
SourceDepot.Sync
ActiveDirectory.User.Create
SourceSafe.Get
ActiveDirectory.User.GrantPrivilege
SourceSafe.LabelGet
AssignCulture
SourceSafe.LabelLatest
BizTalk2002.Configure
SourceSafe.Changes
Cab.AddFile
Sql.DisconnectUsers
Cab.Create
Sql.Execute
Cab.ExtractFile
Sql.Access.Revoke
Certificates.AddCertificate
Sql.Role.AddUser
CheckComponentInstalled
Sql.Access.Grant
CheckProductInstalled
ShortenPath"/>
CodeCoverage.AddAssembly
StringToItemList"/>
CodeCoverage.Create
StringComparison"/>
CodeCoverage.MergeReports
StringReplace"/>
CodeCoverage.Start
Summary.AddToReport
CodeCoverage.Stop
Time.Get
CompileTestSummary
Time.Diff
ComponentServices.Application.AddComponent
Time.Report
ComponentServices.Application.Delete
Tools.DevEnv
ComponentServices.Application.RemoveComponent
Tools.FxCop
ComponentServices.Application.Shutdown
Tools.Installshield
ComponentServices.Application.Update
Tools.MsTest"/>
ComponentServices.Component.Update
Tools.Ndoc
ConsoleReadLine
Tools.Nunit
CreateGuid
Tools.PreSharp
Email
Tools.PsExec
EventSource.Create
Tools.StrongName.AddSkipVerification
EventSource.Log
Tools.StrongName.ReSign
File.Delete
Tools.StyleCop
File.GetFiles
TrimJavascript
File.RegEx
VersionNumber.CreateSourceFiles
Folder.Copy
VersionNumber.Load
Folder.GetInfo
VersionNumber.SplitBuildNumber
Folder.Share.Connect
VersionNumber.Update
Folder.Share.Create
VersionNumber.VSSUpdate
Folder.Share.Delete
VirtualServer.VirtualMachine.OS.CheckHeartBeat
Folder.Share.Disconnect
VirtualServer.VirtualMachine.OS.Shutdown
Folder.Share.Exists
VirtualServer.VirtualMachine.Start
GetInstalledComponents
VirtualServer.VirtualMachine.Stop
GetInstalledProducts
Web.AppPool.Create
GetMetadataValueFromList
Web.AppPool.Delete
GlobalAssemblyCache.AddAssembly
Web.FtpSite.Create
GlobalAssemblyCache.RemoveAssembly
Web.FtpSite.CreateVirtualDirectory
Help.Compile
Web.FtpSite.Delete
Help.CreateProject
Web.FtpSite.DeleteVirtualDirectory
Help.Decompile
Web.FtpSite.Start
Help.DocumentExceptions
Web.FtpSite.Stop
Help.InsertAfter
Web.ServiceExtension.AddFile
Help.InsertBefore
Web.ServiceExtension.DeleteFile
Help.InsertParent
Web.WebSite.AddBinding
LogicalComparison
Web.WebSite.AddFilter
MessageQueue.Create
Web.WebSite.AddHttpCustomHeader
MessageQueue.Delete
Web.WebSite.AddMimeType
MessageQueue.SetPermissions
Web.WebSite.Continue
Msi.EmbedInstallProperties
Web.WebSite.Create
Msi.EmbedUninstallProperties
Web.WebSite.CreateVirtualDirectory
Msi.GetProperty
Web.WebSite.Delete
Msi.Install
Web.WebSite.Modify
Msi.Repair
Web.WebSite.DeleteFilter
Msi.Uninstall
Web.WebSite.DeleteVirtualDirectory
PerformanceCounters.Add
Web.WebSite.FilterExists
PerformanceCounters.Remove
Web.WebSite.Pause
Registry.CreateKey
Web.WebSite.Start
Registry.DeleteKey
Web.WebSite.Stop
Registry.DeleteKeyTree
Web.WebSite.UnloadVirtualDirectory
Registry.Get
Web.WebSite.UpdateHttpErrorSetting
Registry.Set
Web.WebSite.UpdateLogSettings
Security.AddAcl
Web.WebSite.UpdateServerCertificate
Security.RemoveAcl
Web.Smtp.Create
ServiceProcess.Exists
Web.Smtp.Start
ServiceProcess.Start
Web.Smtp.Stop
ServiceProcess.Stop
Wix.CompileMsi
ServiceProcess.UpdateIdentity
Wix.LinkMsi
Sleep
Wix.Fragment
SourceDepot.Changes
Xml.CanonicalizeFile
SourceDepot.ChangesInInterval
Xml.ModifyFile
SourceDepot.CreateBranch
Xml.GetValue
SourceDepot.CreateClientFromTemplate
Xml.XslTransform
SourceDepot.DeleteBranch
Zip.AddFile
SourceDepot.GetChangelistFromDateTime
Zip.Create
SourceDepot.GetChangelistFromLabel
Zip.ExtractFile
SourceDepot.GetUsersForChangelists
MergeByOrder
SourceDepot.Integrate
MergeByRef
SourceDepot.LabelSync
I’m sure that some of these tasks you’ll find completely useless. For example I presume that most of you will not be needing the SourceDepot.* tasks. But there are several very cool tasks that are made available here. Tasks for code coverage, tasks for invoking common tools and other things. I think its great that they have made this available and hope to see some of these tasks integrated into the next realease of MSBuild.
Sayed Ibrahim Hashimi
(Edit: Updated SDC tasks link)
msbuild | Visual Studio
Saturday, March 18, 2006 7:23:16 AM (GMT Standard Time, UTC+00:00)
|
A friend of mine was installing SQL Server 2005 & Visual Studio 2005 on a new machine he received. After installing SQL Server 2005, he tried to install VS 2005 and received an error stating that he couldn't install VS 2005 becase beta versions were present on his machine. I had this problem earlier when I needed to upgrade my machine from RC to RTM. The problem was that he accidently installed SQL Server 2005 CTP. I told him to run the him to download and run the Visual Studio Beta Removal tool. When I had this problem it solved it right up for me. Unfortunately, I think Windows got confused with all the installing/un-installing that was going on. He got his machine to the point where SQL Server wouldn't show up in the add/remove programs list, he couldn't run the un-installer for it because it wasn't present, and he couldn't re-install the same CTP version to uninstall it correctly. And of course he couldn't install VS 2005. It seemed like he was in a real tough spot, we searched around for more information about how to get around this situation. I found a tool, the Windows Installer Clean Up tool, that can be used to removal applications and installer files from failed apps that don't show up in the add/remove programs list. When he ran this tool he was able to see and remove the SQL related components. Following this he was able to install the correct versions of SQL Server 2005 & Visual Studio 2005.
When using MSBuild you can import external declarations into your project by using the Import element. This is very useful for distributing your shared targets across your organization, and is a way to segregate where things are located. When MSBuild encounters this tag, a few things happen. First the current directory is changed to that of the file that is being imported. This is done so assembly locations specified by the Using Task and other imports are resolved correctly. What you may be surprised to find out is that if the importing project file declares any items, those item declarations are always relative to the top project file location. To clarify this if you have the following directory structure:
C:.
├───one
│└───Two
│YourProject.proj
│
├───Shared
││CurrentDirectory.dll
││SharedTargets.targets
││
│└───Another
│SharedTargets_2.targets
│
└───utils
CommandLine.txt
In this scenario YourProject.proj is the top project file, it imports both SharedTargets.targets and SharedTargets_2.targets files. If the SharedTargets.targets had an item declaration of
When imported by YourProject.proj this item declaration would actually resolve to C:\one\two\test.txt instead of the expected C:\Shared\test.txt value. In 95% of the time this is not an issue at all. But for that other 5% how can we accomplish this?
Well, there is no magic reserved MSBuild property for this. So we're gonna have to do some work here. To accomplish this we'll have to create a custom MSBuild task. If you've never created an MSBuild task you might be surprised how easy it is!
I created a new project named DirectoryTask which will house this task. Following this I added a reference to Microsoft.Build.Framework and Microsoft.Build.Utilities assemblies. Following this I wrote the task. It is shown below
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;
namespace CurrentDirectory
{
///<summary>
/// Task that will return the folder that contains the current project.
/// Inheriting from AppDomainIsolatedTask causes this task to execute in its own
/// App domain. Not necessary here, only for demonstration.
///
/// Sayed Ibrahim Hashimi
/// www.sedodream.com
///</summary>
publicclassCurrentDir : AppDomainIsolatedTask
{
privateITaskItem currentDir;
[Output]
publicITaskItem CurrentDirectory
{
get
{
returnthis.currentDir;
}
}
publicoverridebool Execute()
{
System.IO.FileInfo projFile = new System.IO.FileInfo(base.BuildEngine.ProjectFileOfTaskNode);
As you can see this is a pretty simple task. When Execute is called the directory is gathered from the project file that contains the task invocation. The name of the assembly that I built this into is CurrentDirectory.dll.
Now to see this in action I will use the same directory structure shown above. The contents of the SharedTargets.targets file is:
|
VsixCompress
