- | rssFeed | My book on MSBuild and Team Build | Archives and Categories Wednesday, December 28, 2005

MSBuild CreateProperty[Item] and CallTarget bug

It seems like I'm constantly moving files from one place to another, I think this is because I have way too many storage/machine options. (As a side note, my personal storage capacity is at 800GB currently...thats scary.) I wanted to have a means to cleanly and easily snyc up these files. I was considering purchasing (or finding online) a sync tool, but I didn't want to go that route because a friend of mine, Daniel Brookshier, lost about 4 days of work due to an error with the sync tool. I'm not sure what the error was. So I decided to forget about that approach. Instead I wanted to write an MSBuild project file to do the syncing for me. The idea was to have an MSBuild file that I could place in whatever directories that I wanted to snyc, then I could define the various different locations that I wanted to store these files; ie where on my notebook, where on my desktop; where on my removable media X.
Anywayz to make a long story short I discovered a bug in MSBuild! It creeps up when you invoke the CreateProperty or CreateItem, then invoke CallTarget that will reference that newly created property or item. To clariify this have a look at this simple MSBuild project file.

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="Create">
    <
Target Name="Create">
        <
CreateProperty Value="E:\Data">
            <
Output TaskParameter="Value" PropertyName="DestFolder" />
        </
CreateProperty>
        <
CreateItem Include="*.*">
            <
Output TaskParameter="Include" ItemName="TestFiles"/>
        </
CreateItem>
       
       
<CallTarget Targets="Print"/>
    </
Target>

    <Target Name="Print">
        <
Message Text="Dest: $(DestFolder)"/>
        <
Message Text="TestFiles: @(TestFiles)"/>
    </
Target>

</Project>

This file is pretty simple, 2 targets; Create target will create an Item and a Property dynamically then the Print target is called. This will simply print the values the property and item just created. If you have MSBuild invoke the Create target you'll find that this doesn't work. For one reason or another a target called with CallTarget in the same target that properties/items are create in (using CreateProperty or CreateItem) doesn't get the value of those properties/items. I filed a bug report and they will look into fixing this for the next version evidently. You can read up at: http://lab.msdn.microsoft.com/ProductFeedback/viewfeedback.aspx?feedbackid=74e5cb0e-4075-4c7e-9660-c130f94df465 

I've included a zip file containing 2 project files, the one above and one that will show the expected behavior.

sample.zip (.95 KB)

Sayed Ibrahim Hashimi

msbuild | Visual Studio Wednesday, December 28, 2005 5:03:03 AM (GMT Standard Time, UTC+00:00)  #     | 
Monday, December 19, 2005

MSBuild: Extending the clean

When you are creating your own targets, and tasks, they may generate temporary files that need to be cleaned. This is especailly true if you are using third party tools, because alot of times they create a cache directory or give files names that are not agreeable to the developer. When youre MSBuild targets create these files, you may think that if you place the files in the OutputPath then they will be cleaned whenever the Clean is invoked, but this assumption is wrong. Visual Studio actually uses a file that keeps track of what files it generates and only deletes those files. You can have a look at this file its named ProjectName.ProjectType.FileList.txt, For instance the file may be named DataAccess.csproj.FileList.txt. A sample of what this file contains is:
    bin\Debug\Dreamcatcher.DataAccess.dll
    bin\Debug\Dreamcatcher.DataAccess.pdb
    obj\Debug\ResolveAssemblyReference.cache
    obj\Debug\Dreamcatcher.DataAccess.dll
    obj\Debug\Dreamcatcher.DataAccess.pdb
Pretty simple, one line for each file this file is written using the WriteLinesToFile MSBuild task. When the Clean is invoked then this file is read and each of these files are deleted. You could add lines to this file in an attempt to have MSBuild clean them, but this is tricky and you have no control over what happens during the clean. A better way is to create a custom target and inject that into the Clean process. This gives you much more flexibility and is more robust. So let's take a look at how we can do this.

This sample is from a targets file I wrote to invoke xsd.exe against xsd files in certain directories. I might use the rest of that file for later blogs. Here is the relevant portion

<PropertyGroup>
    <
CleanDependsOn>
       
$(CleanDependsOn);
        CleanXsd;
   
</CleanDependsOn>
</
PropertyGroup>

<!--
Target that will remove the xsd generated files if they still exist on disk.
This target will be called whenever the Clean target is invoked.
This target uses dynamically created items using CreateItems instead of normaal Items,
in the case that both the xsd generation targets and the Clean target were invoked sequentially.
For instance: msbuild.exe /t:GenerateAllBindings;Clean
In this case the normal items may not be accurate but dynamic ones should be.
-->
<
Target Name="CleanXsd">
    <
CreateItem Include="$(XsdTempClasses)**\*">
        <
Output TaskParameter="Include" ItemName="TempClassFiles"/>
    </
CreateItem>
    <
Delete Files="@(TempClassFiles)"/>
   
<CreateItem Include="$(XsdTempDataSet)\**.*">
        <
Output TaskParameter="Include" ItemName="TempDSFiles"/>
    </
CreateItem>
    <
Delete Files="@(TempDSFiles)"/>

    <CreateItem Include="$(XsdTempDir)\**.*">
        <
Output TaskParameter="Include" ItemName="OtherFiles"/>
    </
CreateItem>

    <Delete Files="@(OtherFiles)"/>

    <!--Remove the directories-->
    <
RemoveDir Directories="$(XsdTempClasses);$(XsdTempDataSet);$(XsdTempDir)"/>
</
Target>

Pay close attention to the declaration of the CleanDependsOn, this will get the current value of it with $(CleanDependsOn) and append the custom target CleanXsd to the end of it. What's great about doing this is that if you have many different indpendent targets that each need to have items cleaned then they will not affect each other, they will all have have their targets appended to the lits of targets to be executed when a Clean is invoked. If you simply created the AfterClean target in many different targets files then only the last one defined would actually be executed. Also have a look at how the target exclusively uses CreateItem as opposed to items declared in the normal means. Items and Properties are evaluated at the begining of the build. So if I create the Item declaration:

<ItemGroup>
     <TempDSFiles Include="$(XsdTempDataSet)\**.*">

</ItemGroup>

Then if I created Xsd files and cleaned in the same MSBuild instance then I would miss some files. Since it only uses CreateItem for the items then all files will be picked up, no matter when they were created.

msbuild | Visual Studio Monday, December 19, 2005 7:18:24 AM (GMT Standard Time, UTC+00:00)  #     | 
Saturday, December 17, 2005

Clean many projects

There are many occasions that I'd like to clean a large amount of projects at once. Before MSBuild it seems that you would have to do a search/replace to delete all the files/folders that you considered to be "cleanable". Even though this approach would work, you may miss files, or even worse you may delete files that truly shouldn't be deleted. Using MSBuild you can easily implement this, and you don't have to be worried about deleting important files. Let's see how you can do this, if you have a large source tree and you want to delete the "cleanable" then you can use the following MSBuild project file to achieve this.

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
    <
ItemGroup>
        <
Projects Include="**\*.*proj" />
    </
ItemGroup>
    <
Target Name="CleanAllProjects">
       
<MSBuild Projects="@(Projects)" Targets="Clean" StopOnFirstFailure="false" ContinueOnError="true">
       
</MSBuild>
    </
Target>
</
Project>

Place this file at the top of the hierarchy of projects and lets name it for CleanAll.proj. This is about as simple as it gets for a usable MSBuild file. I'll explain this one, in case you're not familiar with MSBuild.

There is a lone Item, Projects, defined in this file and all project files in the current and directories below it will be included in the list. It achieves this by using the **\*.proj include declaration. Then in the CleanAllProjects target the MSBuild task is invoked. We invoke it with the Project = @(Projects). Since the @(Projects) contains many different projects the MSBuild task will be invoked once for each of those projects. The MSBuild task will invoke the Clean target on these project, and since we specified StopOnFirstFailure="false", and ContinueOnError="true" because we want the clean to continue regardless of whether there were any "errors". We can invoke this at the Visual Studio Command Line using >msbuild.exe CleanAll.proj /t:CleanAllprojects. If you are writing MSBuild projects by hand then you'll probably get some "errors" from this process because you probably have project files that don't import Microsoft.Common.targets at some point, so there will be no Clean target defined. You can ignore these "errors".

I think that this simple example demonstrates how the project being the MSBuild file is very important and very convenient. In order to get this sample to work, you don't have to plan for it ahead of time, you don't have to make any modifications to your project. You simply write the code in the same ways that you always have, but now you have another tool available to you for free. This is one thing that makes MSBuild stand out from any other build tools. The fact that they are integrated into the projects enables the developers and engineers to make things happen that typically would be complicated or very in-convenient. With this sample I can be assured that all files are removed correctly, and I don't have to worry about removing files that should remain present. Also if the projects have customizations to the clean those will be included as well. What do you think of this?

Sayed Ibrahim Hashimi

Saturday, December 17, 2005 11:22:41 PM (GMT Standard Time, UTC+00:00)  #     | 
Friday, December 16, 2005

Introduction

Just wanted to introduce myself, I’m Sayed Ibrahim Hashimi. Not to be confused with my brother Sayed Y. Hashimi :)
My brother and I are currently working on an Apress book titled “Deploying .NET Applications with MSBuild and ClickOnce“. I’m focusing on the MSBuild portions of the book while he is mainly working on the ClickOnce related issues.

In this blog you can expect to see me talking about MSBuild, C#,Java and development in general. Also I wouldn’t be surprised to see some comparisions between Windows development and Java development. I might also post about some other things that I'm generally interested, so basically you might see anything in this blog. Currently I’m working professionally as a C# developer but have worked in the past with Java. I think Java is a great language, but I am certainly in favor of C# these days, but each language has it strong points. In the near future I plan on posting some MSBuild related material because I’m really excited about this new technology that is hitting the scenes and I think that it will change the way Windows applications are developed all together. No longer do we have to hook up some 3rd party tool(s) to enhance or replace the Visual Studio build process. I already have some material that is not going to make it into the book, but would like to make it available to everyone, so I’ll post that here!. I would also like to enhance the awareness of how developers can use MSBuild to make their lives easier.

Friday, December 16, 2005 6:09:30 AM (GMT Standard Time, UTC+00:00)  #     |