- | rssFeed | My book on MSBuild and Team Build | Archives and Categories Tuesday, 22 July 2014

Stop checking-in binaries, instead create self-bootstrapping scripts

A few weeks ago Mads Kristensen and I created a few site extensions for Azure Web Sites which the Azure Image Optimizer and Azure Minifier. These extensions can be used to automatically optimize all images on a site, and minify all .js/.css files respectively. These are shipped as nuget packages in nuget.org as well as site extensions in siteextensions.net.

After creating those utilities we also update the image optimizer to support being called in on the command line via a .exe. We have not yet had a chance to update the minifier to be callable directly but we have an open issue on it. If you can help that would be great.

The exe for the image optimizer that can be used from the command line can be found in the nuget package as well. You can also download it from here, but to get the latest version you nuget.org is the way to go.

After releasing that exe I wanted an easy way to use it on a variety of machines, and to make it simple for others to try it out. What I ended up with is what I’m calling a “self-bootstrapping script” which you can find at optimize-images.ps1. Below you’ll see the entire contents of the script.

[cmdletbinding()]
param(
    $folderToOptimize = ($pwd),

    $toolsDir = ("$env:LOCALAPPDATA\LigerShark\tools\"),

    $nugetDownloadUrl = 'http://nuget.org/nuget.exe'
)

<#
.SYNOPSIS
    If nuget is in the tools
    folder then it will be downloaded there.
#>
function Get-Nuget(){
    [cmdletbinding()]
    param(
        $toolsDir = ("$env:LOCALAPPDATA\LigerShark\tools\"),

        $nugetDownloadUrl = 'http://nuget.org/nuget.exe'
    )
    process{
        $nugetDestPath = Join-Path -Path $toolsDir -ChildPath nuget.exe
        
        if(!(Test-Path $nugetDestPath)){
            'Downloading nuget.exe' | Write-Verbose
            (New-Object System.Net.WebClient).DownloadFile($nugetDownloadUrl, $nugetDestPath)

            # double check that is was written to disk
            if(!(Test-Path $nugetDestPath)){
                throw 'unable to download nuget'
            }
        }

        # return the path of the file
        $nugetDestPath
    }
}

<#
.SYNOPSIS
    If the image optimizer in the .ools
    folder then it will be downloaded there.
#>
function GetImageOptimizer(){
    [cmdletbinding()]
    param(
        $toolsDir = ("$env:LOCALAPPDATA\LigerShark\tools\"),
        $nugetDownloadUrl = 'http://nuget.org/nuget.exe'
    )
    process{
        
        if(!(Test-Path $toolsDir)){
            New-Item $toolsDir -ItemType Directory | Out-Null
        }

        $imgOptimizer = (Get-ChildItem -Path $toolsDir -Include 'ImageCompressor.Job.exe' -Recurse)

        if(!$imgOptimizer){
            'Downloading image optimizer to the .tools folder' | Write-Verbose
            # nuget install AzureImageOptimizer -Prerelease -OutputDirectory C:\temp\nuget\out\
            $cmdArgs = @('install','AzureImageOptimizer','-Prerelease','-OutputDirectory',(Resolve-Path $toolsDir).ToString())

            'Calling nuget to install image optimzer with the following args. [{0}]' -f ($cmdArgs -join ' ') | Write-Verbose
            &(Get-Nuget -toolsDir $toolsDir -nugetDownloadUrl $nugetDownloadUrl) $cmdArgs | Out-Null
        }

        $imgOptimizer = Get-ChildItem -Path $toolsDir -Include 'ImageCompressor.Job.exe' -Recurse | select -first 1
        if(!$imgOptimizer){ throw 'Image optimizer not found' }       

        $imgOptimizer
    }
}

function OptimizeImages(){
    [cmdletbinding()]
    param(
        [Parameter(Mandatory=$true,Position=0)]
        $folder,
        $toolsDir = ("$env:LOCALAPPDATA\LigerShark\tools\"),
        $nugetDownloadUrl = 'http://nuget.org/nuget.exe'
    )
    process{        
        [string]$imgOptExe = (GetImageOptimizer -toolsDir $toolsDir -nugetDownloadUrl $nugetDownloadUrl)

        [string]$folderToOptimize = (Resolve-path $folder)

        'Starting image optimizer on folder [{0}]' -f $folder | Write-Host
        # .\.tools\AzureImageOptimizer.0.0.10-beta\tools\ImageCompressor.Job.exe --folder M:\temp\images\opt\to-optimize
        $cmdArgs = @('--folder', $folderToOptimize)

        'Calling img optimizer with the following args [{0} {1}]' -f $imgOptExe, ($cmdArgs -join ' ') | Write-Host
        &$imgOptExe $cmdArgs

        'Images optimized' | Write-Host
    }
}

OptimizeImages -folder $folderToOptimize -toolsDir $toolsDir -nugetDownloadUrl $nugetDownloadUrl

The script is setup to where you call functions like Get-NuGet and GetImageOptimzer to get the path to the .exe to call. If the .exe is not in the expected location, under %localappdata% by default, it will be downloaded and then the path will be returned. In the case of this script I use nuget.org as my primary distribution mechanism for this so the script will first download nuget.exe and then use that to get the actual binaries. WIth this approach, you can avoid checking in binaries and have scripts which are still pretty concise.

After creating optimize-images.ps1 I thought it would be really useful to have a similar script to execute XDT transforms on xml files. So I created transform-xml.ps1. That script first downloads nuget.exe and then uses that to download the nuget packages which are required to invoke XDT transforms.

A self-bootstrapping script doesn’t need to be a PowerShell script, you can apply the same techniques to any scripting language. I’ve recently created an MSBuild script, inspired by Get-Nuget above, which can be used in a similar way. You can find that script in a gist here. It’s below as well.

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="GetNuget">

  <!--
  This is an MSBuild snippet that can be used to download nuget to the path
  in the property NuGetExePath property.
  
  Usage:
   1. Import this file or copy and paste this into your build script
   2. Call the GetNuGet target before you use nuget.exe from $(NuGetExePath)
  -->
  
  <PropertyGroup>
    <NuGetExePath Condition=" '$(NuGetExePath)'=='' ">$(localappdata)\LigerShark\AzureJobs\tools\nuget.exe</NuGetExePath>
    <NuGetDownloadUrl Condition=" '$(NuGetDownloadUrl)'=='' ">http://nuget.org/nuget.exe</NuGetDownloadUrl>
  </PropertyGroup>
  
  <Target Name="GetNuget" Condition="!Exists('$(NuGetExePath)')">
    <Message Text="Downloading nuget from [$(NuGetDownloadUrl)] to [$(NuGetExePath)]" Importance="high"/>
    <ItemGroup>
      <_nugetexeitem Include="$(NuGetExePath)" />
    </ItemGroup>
    <MakeDir Directories="@(_nugetexeitem->'%(RootDir)%(Directory)')"/>
    <DownloadFile
        Address="$(NuGetDownloadUrl)"
        FileName="$(NuGetExePath)" />    
  </Target>

  <PropertyGroup Condition=" '$(ls-msbuildtasks-path)'=='' ">
    <ls-msbuildtasks-path>$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll</ls-msbuildtasks-path>
    <ls-msbuildtasks-path Condition=" !Exists('$(ls-msbuildtasks-path)')">$(MSBuildFrameworkToolsPath)\Microsoft.Build.Tasks.v4.0.dll</ls-msbuildtasks-path>
    <ls-msbuildtasks-path Condition=" !Exists('$(ls-msbuildtasks-path)')">$(windir)\Microsoft.NET\Framework\v4.0.30319\Microsoft.Build.Tasks.v4.0.dll</ls-msbuildtasks-path>
  </PropertyGroup>
  
  <UsingTask TaskName="DownloadFile" TaskFactory="CodeTaskFactory" AssemblyFile="$(ls-msbuildtasks-path)">
    <!-- http://stackoverflow.com/a/12739168/105999 -->
    <ParameterGroup>
      <Address ParameterType="System.String" Required="true"/>
      <FileName ParameterType="System.String" Required="true" />
    </ParameterGroup>
    <Task>
      <Reference Include="System" />
      <Code Type="Fragment" Language="cs">
        <![CDATA[
            new System.Net.WebClient().DownloadFile(Address, FileName);
        ]]>
      </Code>
    </Task>
  </UsingTask>
  
</Project>

This script has a single target, GetNuGet, which you can call to download nuget.exe to the expected location. After that you can use the path to nuget.exe from the NuGetExePath property. I’ve already removed nuget.ext from SideWaffle and AzureJobs repository using this technique. It’s a great way to avoid checking in nuget.exe.

 

Thanks,
Sayed Ibrahim Hashimi | http://msbuildbook.com | @SayedIHashimi

msbuild | powershell | scripting Tuesday, 22 July 2014 07:36:11 (GMT Daylight Time, UTC+01:00)  #     |