Archive for the ‘Build’ Category

PowerShell Log Levels Included In TFS 2017 and VSTS Build and Release Logs

October 26th, 2017 No comments

We use quite a few custom PowerShell scripts in some of our builds and releases. This led me to ask the question, which PowerShell log levels actually get written to the TFS Build and Release logs? So I did a quick test on both our on-premise TFS 2017 Update 2 installation and my personal VSTS account, and they yielded the same results.

The PowerShell Script Used To Test

I created a blank build definition and release definition, and the only thing I added to them was a PowerShell task with the following inline script:

Write-Host "Host"
Write-Output "Output"
Write-Debug "Debug"
Write-Debug "Debug Forced" -Debug
Write-Information "Information"
$InformationPreference = 'Continue'
Write-Information "Information Forced"
Write-Verbose "Verbose"
Write-Verbose "Verbose Forced" -Verbose
Write-Warning "Warning"
Write-Error "Error"
throw "Throw"

The Results

Both the build and the release logs yielded the same results. The real-time output shown in the Console resulted in:

DEBUG: Debug Forced
Write-Debug : Windows PowerShell is in NonInteractive mode. Read and Prompt functionality is not available.
Information Forced
VERBOSE: Verbose Forced
WARNING: Warning
C:\Builds\_work\_temp\c0558237-7d53-43ca-97bf-90ed03b8247f.ps1 : Error

The more detailed information written to the log files was:

2017-10-26T06:31:09.5118196Z Host
2017-10-26T06:31:09.5138198Z Output
2017-10-26T06:31:09.5228183Z DEBUG: Debug Forced
2017-10-26T06:31:09.6678192Z ##[error]Write-Debug : Windows PowerShell is in NonInteractive mode. Read and Prompt functionality is not available.
At D:\a\_temp\3b089729-e7c5-484d-aa58-256b33f12e01.ps1:4 char:1
+ Write-Debug "Debug Forced" -Debug
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Write-Debug], PSInvalidOperationException
    + FullyQualifiedErrorId : InvalidOperation,Microsoft.PowerShell.Commands.WriteDebugCommand

2017-10-26T06:31:09.6698195Z Information Forced
2017-10-26T06:31:09.6698195Z VERBOSE: Verbose Forced
2017-10-26T06:31:09.6698195Z WARNING: Warning
2017-10-26T06:31:09.7258181Z ##[error]D:\a\_temp\3b089729-e7c5-484d-aa58-256b33f12e01.ps1 : Error
At line:1 char:1
+ . 'd:\a\_temp\3b089729-e7c5-484d-aa58-256b33f12e01.ps1'
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,3b089729-e7c5-484d-aa58-256b33f12e01.p 
At D:\a\_temp\3b089729-e7c5-484d-aa58-256b33f12e01.ps1:12 char:1
+ throw "Throw"
+ ~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Throw:String) [], RuntimeException
    + FullyQualifiedErrorId : Throw

One thing to note is that while the throw statements are not shown in the real-time console output, they do show up in the detailed logs that are available once the task step completes.

It’s worth pointing out that even when toggling the system.debug variable to true, regular Debug statements were not written; only the Forced Debug statement was. Also, notice that even though the forced Debug statement was written to the log, it resulted in an error since the script is running in non-interactive mode, so you should probably avoid forcing Debug statements.


So there you have it. By default only the following statements are written to the log files:

  • Write-Host
  • Write-Output
  • Write-Warning
  • Write-Error

and if you force them, you can also have the following written to the log files as well:

  • Write-Debug
  • Write-Information
  • Write-Verbose

And lastly, throw statements do not show up in the real-time console output, but do show up in the detailed log information.

I hope you find this information useful. Happy coding!

Continuously Deploy Your ClickOnce Application From Your Build Server

January 10th, 2017 12 comments

ClickOnce applications are a great and easy way to distribute your applications to many users, and have the advantage of offering automatic application updates out-of-the-box. Even though ClickOnce applications are super easy to deploy from Visual Studio (literally 3 clicks, just click Build –> Publish –> Finish), you may still want to have your build system publish the updates for various reasons, such as:

  1. You don’t want you (or your team members) to have to remember to manually publish a new version all the time; even if it is very quick and easy.
  2. If you are signing your ClickOnce application (to make your app more secure and avoid annoying Windows security warnings during every update), your team members may not have the certificate installed, or you may not want them to know the certificate password.
  3. All of the other benefits of deploying automatically from a build server; you can be sure the project was built in Release mode, all unit tests have run and passed, you are only publishing a new version of the application from a specific branch (e.g. master), etc.

In this post I’m going to show how to continuously deploy your ClickOnce application using Visual Studio Team Services (VSTS), but you can adopt what’s shown here to work on any build system, such as Team City or Jenkins.


Step 1 – Configure and publish your ClickOnce application manually

Before we can publish a new version of the application from the build server, we first have to build the project to create the artifacts to publish. And even before that, we have to make sure the project is setup properly with all of the required ClickOnce metadata. You will typically do this by going into your project Properties page and going to the Publish tab. There are several other websites and blog posts that discuss configuring a project for ClickOnce deployment, including the official MSDN documentation for configuring and publishing ClickOnce applications, so I won’t go into it any further here.

Basically you should have your project in a state where you can easily publish a new version manually, and it is configured the way that you want (includes the necessary files, has the destination to publish to specified, specifies if the application is only available online or not, etc.). Once you’ve published your project manually and confirmed that it’s configured the way you like, we can move onto the next step.


Step 2 – Setup the build on your build server

On your build server you will want to configure a new vanilla build that builds your project/solution. There are a couple modifications you will need to make that are different from building a regular console app or class library project.

The first difference is that you will need to provide the “/target:Publish” argument to MSBuild when it builds your project. Here is what this looks like in VSTS:

This will cause MSBuild to build the required artifacts into an “app.publish” directory. e.g. bin\Debug\app.publish.

The next difference is that you will want to copy that “app.publish” directory to your build artifacts directory. To do this, you will need to add a Copy Files step into your build process that copies the “app.publish” directory from the ClickOnce project’s bin directory to where the build artifacts are expected to be. You will want to do this before the step that publishes your build artifacts. Here is what this looks like in VSTS:

So we copy the files into the build artifacts directory, and then the Publish Build Artifacts step at the end will copy those files to wherever you’ve specified; in my case it’s a network share.

If you like you can now run the build and see if it succeeds. If the build fails with an error relating to an expired certificate or pfx file, see my other blog post on importing the required certificate on the build server at build-time, which involves adding one more “Import-PfxCertificate.ps1” build step before the MSBuild step.

For completeness sake, this is what my Publish Build Artifacts step looks like in VSTS, and you’ll also notice the “Import-PfxCertificate.ps1” step before the MSBuild step as well:

So we now have the ClickOnce artifacts being generated and stored in the appropriate directory. If you wanted, you could publish the build artifacts to the ClickOnce application’s final destination right now (instead of a file share as I’ve done here), but I’m going to follow best practices and separate the application “build” and “deployment” portions into their respective subsystems, as you may want separate control over when a build gets published, or maybe you don’t want to publish EVERY build; only some of them.

Hopefully at this point you are able to create a successful build, but we’re not done yet.


Step 3 – Publish the build artifacts to the ClickOnce application’s destination

Now that we have the build artifacts safely stored, we can publish them to the ClickOnce application’s destination. With VSTS this is done by using the Release subsystem. So create a new Release Definition and setup an Environment for it. By default it adds a “Copy and Publish Build Artifacts” step to the release definition. When configuring and using this default step, I received the error:

[error]Copy and Publish Build Artifacts task is not supported within Release

So instead I removed that default step and added a simple “Copy Files” step, and then configured it to grab the files from the file share that the build published the artifacts to, and set the destination to where the ClickOnce application was publishing to when I would do a manual publish from within Visual Studio. Here is what that step looks like in VSTS:

You should be able to run this release definition and see that it is able to post a new version of your ClickOnce application. Hooray! I setup my releases to automatically publish on every build, but you can configure yours however you like.

If you make changes and commit them, the build should create new artifacts, and the release should be able to publish the new version. However, if you launch your ClickOnce application, you may be surprised that it doesn’t actually update to the latest version that you just committed and published. This is because we need one additional build step to actually update the ClickOnce version so it can detect that a new version was published. If you launch another build and publish them, you’ll see that the files being published are overwriting the previous files, instead of getting published to a new directory as they are supposed to be.

You may be wondering why we need another build step to update the ClickOnce version. Part of the build artifacts that are created are a .application manifest file, and a directory that contains the applications files, both of which have the ClickOnce version in them. Can’t we just modify the directory name and .application manifest file to increment the old version number? This was my initial thought, but the .application manifest file contains a cryptography hash to ensure that nobody has tampered with it, meaning that it needs to contain the proper version number at the time that it is generated.


Step 4 – One more build step to update the ClickOnce version

The ClickOnce version is defined in your project file (.csproj/.vbproj) (as the <ApplicationVersion> and <ApplicationRevision> xml elements), and is different from the assembly version that you would typically set to version your .dll/.exe files via the AssemblyInfo.cs file. Unless you manually updated the ClickOnce version in Visual Studio, it will still have its original version. When publishing the ClickOnce version in Visual Studio, Visual Studio automatically takes care of incrementing ClickOnce version’s Revision part. However, because we’re publishing from our build system now, we’ll need to come up with an alternative method. To help solve this problem, I have created the Set-ProjectFilesClickOnceVersion GitHub repository. This repository houses a single Set-ProjectFilesClickOnceVersion.ps1 PowerShell script that we will want to download and add into our build process.

In order for the build system to access this Set-ProjectFilesClickOnceVersion.ps1 script you will need to check it into source control. Also, you need to make sure you add this build step before the MSBuild step. Here is what this step looks like in VSTS:

The Script Filename points to the location of the Set-ProjectFilesClickOnceVersion.ps1 script in my repository. Also note that I set the Working Folder to the directory that contains the .csproj project file. This is necessary since we are dealing with relative paths, not absolute ones.

For the Arguments I provide the name of the project file that corresponds to my ClickOnce application, as well as the VSTS Build ID. The Build ID is a globally unique ID that gets auto-incremented with every build, and most (all?) build systems have this. The script will translate this Build ID into the Build and Revision version number parts to use for the ClickOnce version, to ensure that they are always increasing in value. So you should never need to manually modify the ClickOnce version, and every build should generate a version number greater than the last, allowing the ClickOnce application to properly detect when a newer version has been published.

Maybe you update your application’s version number on every build, and want your ClickOnce version to match that version. If this is the case, then use the –Version parameter instead of –BuildSystemsBuildId. The other alternative is to use the –IncrementProjectFilesRevision switch to simply increment the ClickOnce Revision that is already stored in the project file. If you use this switch, you need to be sure to check the project file back into source control so that the Revision is properly incremented again on subsequent builds. I like to avoid my build system checking files into source control wherever possible, so I have opted for the BuildSystemsBuildId parameter here.

The last argument I have included is the UpdateMinimumRequiredVersionToCurrentVersion parameter, and it is optional. If provided, this parameter will change the ClickOnce application’s Minimum Required Version to be the same as the new version, forcing the application to always update to the latest version when it detects that an update is available. I typically use this setting for all of my ClickOnce applications. If you still plan on publishing your ClickOnce applications from Visual Studio, but want this functionality, simply install the AutoUpdateProjectsMinimumRequiredClickOnceVersion NuGet Package.

That’s it! Now if you launch another build it should create new artifacts with a larger ClickOnce version number, and once published your application should update to the new version.


Bonus – Displaying the ClickOnce version in your application

The ClickOnce application version number may be different than your assembly version number, or perhaps you don’t update your product’s version number on every publish; it’s easy to forget. I typically like to use semantic versioning for my projects, which only involves the first 3 version parts. This leaves the last version part, the Revision number, available for me to set as I please. I will typically use the ClickOnce Revision in my application’s displayed version number. This makes it easy to tell which actual release a client has installed in the event that the product version was not updated between releases. The code to do it is fairly simple.

public Version ApplicationVersion = new Version("1.11.2");

private void MainWindow_Loaded(object sender, RoutedEventArgs e)
	// If this is a ClickOnce deployment, append the ClickOnce Revision to the version number (as it gets updated with each publish).
	if (ApplicationDeployment.IsNetworkDeployed)
		ApplicationVersion = new Version(ApplicationVersion.Major, ApplicationVersion.Minor, ApplicationVersion.Build, ApplicationDeployment.CurrentDeployment.CurrentVersion.Revision);

	// Display the Version as part of the window title.
	wndMainWindow.Title += ApplicationVersion;

Here I hard-code the product version that I want displayed in my application in a variable called ApplicationVersion. When the WPF application is launched, I obtain the ClickOnce Revision and append it onto the end of the version number. I then display the version in my application’s window title, but you might want to show it somewhere else, such as in an About window. If you want, you could even display both the full application version and full ClickOnce version.


I hope this blog has helped you further your automation-foo. Happy coding!

Creating A .PFX Certificate And Applying It On The Build Server At Build Time

December 1st, 2016 10 comments

There are many project types that require a .pfx file in order to build and/or publish successfully, such as ClickOnce and UWP (Universal Windows Platform) applications. When you build these projects locally in Visual Studio everything is fine and dandy, but when you try to build them on a build server (such as part of a Continuous Integration build) the build server may fail with an error like:

Error APPX0108: The certificate specified has expired. For more information about renewing certificates, see

error MSB4018: The “ResolveKeySource” task failed unexpectedly.
System.InvalidOperationException: Showing a modal dialog box or form when the application is not running in UserInteractive mode is not a valid operation. Specify the ServiceNotification or DefaultDesktopOnly style to display a notification from a service application.

Cannot import the following key file: companyname.pfx. The key file may be password protected.

These errors will often occur when you are still using the temporary file that was generated automatically when the project was created in Visual Studio, or when using a .pfx file that is password protected.

If your .pfx file is password protected, we can import the certificate at build time, as shown further below. The nice thing about this approach is it does not require you (or one of your company admins) to manually install the certificate on every build server, as many other sites suggest. This way you don’t have to bother your admin or get unexpected broken builds when a new build server is added to the pool.

If your .pfx file has expired, you need to remove the current pfx file and add a new (password-protected) one in its place.


Creating a new .pfx file for a UWP application

To create a new password protected .pfx file in a UWP application,

1. open the Package.appxmanifest file that resides in the root of the project and

2. go to the Packaging tab.

3. In here, click the Choose Certificate… button.

4. In the Configure Certificate… dropdown, choose an existing certificate that you have, or create a new test certificate and provide a password for it. This should create the certificate and add the new .pfx file to your project’s root.

Create Pfx Certificate In UWP App


Creating a new .pfx file for a ClickOnce application

Creating a pfx file for a ClickOnce application is similar, but instead you want to

1. open up the project’s Properties window (by right-clicking on the project) and

2. go to the Signing tab.

3. Check off the box to Sign the ClickOnce manifests and then

4. choose the certificate you want to use, or create your own password-protected test certificate.

Create Pfx Certificate In ClickOnce App


Applying the .pfx file before building on the build server

Now that the project has a pfx certificate, we will need to update our build server to apply it before attempting to build the solution, otherwise a certificate related error will likely be thrown.

Before building the solution, we will want to apply the certificate using this PowerShell script, Import-PfxCertificate.ps1:

param($PfxFilePath, $Password)

$absolutePfxFilePath = Resolve-Path -Path $PfxFilePath
Write-Output &quot;Importing store certificate &#39;$absolutePfxFilePath&#39;...&quot;

Add-Type -AssemblyName System.Security
$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$cert.Import($absolutePfxFilePath, $Password, [System.Security.Cryptography.X509Certificates.X509KeyStorageFlags]::PersistKeySet)
$store = new-object -argumentlist &quot;MY&quot;, CurrentUser

You can download the Import-PfxCertificate Powershell script from here.

To do this you will need to save this Import-PfxCertificate.ps1 file in your version control repository so that the build system has access to it. In my projects I have a root buildTools directory that I store these types of build scripts in. In your build system, you will need to configure a new step to run the PowerShell script before the step to actually build your solution. When you call the script you will need to pass it the path to the .pfx file (which should also be checked into source control), and the password for the .pfx file. If your build system supports using “hidden”/”private” variables, then it is recommended you use them to keep the .pfx file password a secret.

Here I am using VSTS (Visual Studio Team Services), but the same steps should generally apply to any build system (e.g. TeamCity). I have created a new build step called Apply Store Certificate that calls the Import-PfxCertificate.ps1 PowerShell script. This step is set to occur before the Build solution step, and provides the required parameters to the script; the path to the .pfx file and the certificates password (as a hidden variable that I configured in the build system). Notice that I also set the Working folder that the script will run from to the directory that the .pfx resides in, so that the script will be able to resolve the absolute file path.

VSTS Build Pfx Certificate Step

Your build system should now be able to apply the certificate before building the solution, avoiding the certificate errors and hopefully resulting in a successful build.

Hopefully this post has helped you out. Happy coding!

Fix MSBuild 2015 Compile Taking A Very Long Time

April 14th, 2016 1 comment

I created the Invoke-MsBuild PowerShell Module (also available in the PowerShell Gallery), and recently added support to use the Visual Studio 2015 version of MsBuild.exe, when available. After doing so, I noticed that sometimes the build would take a very long time to complete; a solution that typically would take 10 seconds to compile was all of a sudden taking 10 minutes. When changing Invoke-MsBuild back to defaulting to the Visual Studio 2013 version of MsBuild.exe the problem went away. I thought that maybe there was just something strange with my workstation, however after updating Invoke-MsBuild on our build servers at my work we saw the same thing there.

Luckily, a fellow by the name of Jens Doose contacted me saying that he was experiencing the same problem when using Invoke-MsBuild, and also that he had fixed it. The solution ended up being that when calling MsBuild.exe, we had to specify an additional command-line argument of /p:UseSharedConfiguration=false.

msbuild.exe someSolution.sln /p:Configuration=Release /p:UseSharedConfiguration=false

I’m not sure what the other implications of providing this UseSharedConfiguration parameter are as I can’t find any documentation of it online, and I’m not really sure how Jens came across it. It does seem to solve the problem of compiling take a long time though, and I haven’t noticed any other side effects, so I’m going to stick with it.

If you run into the same problem with msbuild.exe and this helps you out, leave a comment to let me know. Happy coding!

“Agent lost communication with Team Foundation Server” TFS Build Server Error

March 12th, 2014 1 comment

We had recently started getting lots of error messages similar to the following on our TFS Build Servers:

Exception Message: The build failed because the build server that hosts build agent TFS-BuildController001 - Agent4 lost communication with Team Foundation Server. (type FaultException`1) 

This error message would appear randomly; some builds would pass, others would fail, and when they did fail with this error message it was often at different parts in the build process.

After a bit of digging I found this post and this one, which discussed different error messages around their build process failing with some sort of error around the build controller losing connection to the TFS server.  They talked about different fixes relating to DNS issues and load balancing, so we had our network team update our DNS records and flush the cache, but were still getting the same errors.

We have several build controllers, and I noticed that the problem was only happening on two of the three, so our network team updated the hosts file on the two with the problem to match the entries in the one that was working fine, and boom, everything started working properly again 🙂

So the problem was that the hosts file on those two build controller machines somehow got changed.

The hosts file can typically be found at "C:\Windows\System32\Drivers\etc\hosts", and here is an example of what we now have in our hosts file for entries (just the two entries):

12.345.67.89	TFS-Server.OurDomain.local
12.345.67.89	TFS-Server

If you too are running into this TFS Build Server error I hope this helps.

Automatically Create Your Project’s NuGet Package Every Time It Builds, Via NuGet

June 22nd, 2013 15 comments

So you’ve got a super awesome library/assembly that you want to share with others, but you’re too lazy to actually use NuGet to package it up and upload it to the gallery; or maybe you don’t know how to create a NuGet package and don’t have the time or desire to learn.  Well, my friends, now this can all be handled for you automatically.

A couple weeks ago I posted about a new PowerShell script that I wrote and put up on CodePlex, called New-NuGetPackage PowerShell Script, to make creating new NuGet packages quick and easy.  Well, I’ve taken that script one step further and use it in a new NuGet package called Create New NuGet Package From Project After Each Build (real creative name, right) that you can add to your Visual Studio projects.  The NuGet package will, you guessed it, pack your project and its dependencies up into a NuGet package (i.e. .nupkg file) and place it in your project’s output directory beside the generated dll/exe file.  Now creating your own NuGet package is as easy as adding a NuGet package to your project, which if you’ve never done before is dirt simple.

I show how to add the NuGet package to your Visual Studio project in the New-NuGetPackage PowerShell Script documentation (hint: search for “New NuGet Package” (include quotes) to find it in the VS NuGet Package Manager search results), as well as how you can push your package to the NuGet Gallery in just a few clicks.

Here’s a couple screenshots from the documentation on installing the NuGet Package:

NavigateToManageNugetPackages   InstallNuGetPackageFromPackageManager

Here you can see the new PostBuildScripts folder it adds to your project, and that when you build your project, a new .nupkg file is created in the project’s Output directory alongside the dll/exe.

FilesAddedToProject     NuGetPackageInOutputDirectory

So now that packaging your project up in a NuGet package can be fully automated with about 30 seconds of effort, and you can push it to the NuGet Gallery in a few clicks, there is no reason for you to not share all of the awesome libraries you write.

Happy coding!

Using MSBuild to publish a VS 2012 SSDT .sqlproj database project the same way as a VS 2010 .dbproj database project (using command line arguments to specify the database to publish to)

March 18th, 2013 20 comments

Post and code updated on March 21, 2013, and again on March 22, 2013.

We recently upgraded from VS (Visual Studio) 2010 to VS 2012, and with it had to upgrade our .dbproj database project to a .sqlproj.  When making the switch I realized that .sqlproj database projects do not support specifying the database to deploy to as MSBuild command line arguments; instead you have to pass in the path to an xml file that has the necessary information.

So with the old .dbproj database project, you could deploy it to a database using:

MSBuild /t:Deploy /p:TargetDatabase="[DbName]";TargetConnectionString="Data Source=[Db.Server];Integrated Security=True;Pooling=False" /p:DeployToDatabase="True" "[PathToBranch]Database\Database.dbproj"

But with the new .sqlproj database project you have to do:

MSBuild /t:Publish /p:SqlPublishProfilePath="myPublishFile.publish.xml" "[PathToBranch]Database\Database.sqlproj"

Where “myPublishFile.publish.xml” contains the database server and name to publish to.

One other minor thing to note is that it is called “deploying” the database with .dbproj, and is called “publishing” the database with .sqlproj; so when I say Deploy or Publish, I mean the same thing.

We use TFS at my organization and while making new builds for our Test environment, we have the build process deploy the database solution to our various Test databases.  This would mean that for us I would either need to:

1 – create a new [DbName].publish.xml file for each database, check it into source control, and update the build template to know about the new file, or

2 – update the file contents of our myPublishFile.publish.xml file dynamically during the build to replace the Database Name and Server in the file before publishing to the database (i.e. read in file contents, replace string, write file contents back to file, publish to DB, repeat).

Option 1 means more work every time I want to add a new Test database to publish to.  Option 2 is better, but still means having to update my TF Build template and create a new activity to read/write the new contents to the file.

Instead, there is a 3rd option, which is to simply add the code below to the bottom of the .sqlproj file.  This will add some new MSBuild targets to the .sqlproj that will allow us to specify the database name and connection string using similar MSBuild command line parameters that we used to deploy the .dbproj project.

The code presented here is based on this post, but the author has closed the comments section on that post and has not replied to my emails about the bugs in his code and example, so I thought I would share my modified and enhanced solution.

	Custom targets and properties added so that we can specify the database to publish to using command line parameters with VS 2012 .sqlproj projects, like we did with VS 2010 .dbproj projects.
	This allows us to specify the MSBuild command-line parameters TargetDatabaseName, and TargetConnectionString when Publishing, and PublishToDatabase when Building.
	I also stumbled across the undocumented parameter, PublishScriptFileName, which can be used to specify the generated sql script file name, just like DeployScriptFileName used to in VS 2010 .dbproj projects.
	Taken from:
  <PropertyGroup Condition="'$(TargetDatabaseName)' != '' Or '$(TargetConnectionString)' != ''">
    <PublishToDatabase Condition="'$(PublishToDatabase)' == ''">False</PublishToDatabase>
    <TargetConnectionStringXml Condition="'$(TargetConnectionString)' != ''">
      &lt;TargetConnectionString xdt:Transform="Replace"&gt;$(TargetConnectionString)&lt;/TargetConnectionString&gt;
    <TargetDatabaseXml Condition="'$(TargetDatabaseName)' != ''">
      &lt;TargetDatabaseName xdt:Transform="Replace"&gt;$(TargetDatabaseName)&lt;/TargetDatabaseName&gt;
        &lt;Project xmlns:xdt="" xmlns=""&gt;
    <SqlPublishProfilePath Condition="'$([System.IO.Path]::IsPathRooted($(SqlPublishProfilePath)))' == 'False'">$(MSBuildProjectDirectory)\$(SqlPublishProfilePath)</SqlPublishProfilePath>
    <!-- In order to do a transform, we HAVE to change the SqlPublishProfilePath -->
    <TransformStackTraceEnabled Condition="'$(TransformStackTraceEnabled)'==''">False</TransformStackTraceEnabled>
  <Target Name="AfterBuild" Condition="'$(PublishToDatabase)'=='True'">
    <CallTarget Targets="Publish" />
  <UsingTask TaskName="ParameterizeTransformXml" AssemblyFile="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\v$(VisualStudioVersion)\Web\Microsoft.Web.Publishing.Tasks.dll" />
  <Target Name="BeforePublish" Condition="'$(TargetDatabaseName)' != '' Or '$(TargetConnectionString)' != ''">
    <Message Text="TargetDatabaseName = '$(TargetDatabaseName)', TargetConnectionString = '$(TargetConnectionString)', PublishScriptFileName = '$(PublishScriptFileName)', Transformed Sql Publish Profile Path = '$(TransformOutputFile)'" Importance="high" />
    <!-- If TargetDatabaseName or TargetConnectionString, is passed in then we use the tokenize transform to create a parameterized sql publish file -->
    <Error Condition="!Exists($(SqlPublishProfilePath))" Text="The SqlPublishProfilePath '$(SqlPublishProfilePath)' does not exist, please specify a valid file using msbuild /p:SqlPublishProfilePath='Path'" />
    <ParameterizeTransformXml Source="$(SqlPublishProfilePath)" IsSourceAFile="True" Transform="$(TransformPublishXml)" IsTransformAFile="False" Destination="$(TransformOutputFile)" IsDestinationAFile="True" Scope="$(TransformScope)" StackTrace="$(TransformStackTraceEnabled)" SourceRootPath="$(MSBuildProjectDirectory)" />


So after adding this code at the bottom of the .sqlproj file (above the </Project> tag though), you can now build and publish the database solution from the MSBuild command line using:

MSBuild /t:Build /p:TargetDatabaseName="[DbName]";TargetConnectionString="Data Source=[Db.Server];Integrated Security=True;Pooling=False" /p:PublishToDatabase="True" /p:SqlPublishProfilePath="Template.publish.xml" "[PathToBranch]\Database\Database.sqlproj"

Here you can see the 3 new parameters that we’ve added being used: TargetDatabaseName, TargetConnectionString, and PublishToDatabase.

When the TargetDatabaseName or TargetConnectionString parameters are provided we generated a new transformed .publish.xml file, which is the same as the provided “Template.publish.xml” file, but with the database and connection string values replaced with the provided values.

The PublishToDatabase parameter allows us to publish to the database immediately after the project is built; without this you would have to first call MSBuild to Build the database project, and then call MSBuild again to Publish it (or perhaps using “/t:Build;Publish” would work, but I didn’t test that).

If you want to simply publish the database project without building first (generally not recommended), you can do:

MSBuild /t:Publish /p:TargetDatabaseName="[DbName]";TargetConnectionString="Data Source=[Db.Server];Integrated Security=True;Pooling=False" /p:SqlPublishProfilePath="Template.publish.xml" "[PathToBranch]\Database\Database.sqlproj"

Be careful though, since if you don’t do a Build first, any changes that have been made since the last time the .sqlproj file was built on your machine won’t be published to the database.

Notice that I still have to provide a path to the template publish.xml file to transform, and that the path to this file is relative to the .sqlproj file (in this example the Template.publish.xml and .sqlproj files are in the same directory).  You can simply use one of the publish.xml files generated by Visual Studio, and then the TargetDatabaseName and TargetConnectionString xml element values will be replaced with those given in the command line parameters.  This allows you to still define any other publish settings as usual in the xml file.

Also notice that the PublishToDatabase parameter is only used when doing a Build, not a Publish; providing it when doing a Publish will not hurt anything though.

While creating my solution, I also accidentally stumbled upon what seems to be an undocumented SSDT parameter, PublishScriptFileName.  While the DeployScriptFileName parameter could be used in VS 2010 .dbproj projects to change the name of the generated .sql file, I noticed that changing its value in the .publish.xml file didn’t seem to have any affect at all (so I’m not really sure why Visual Studio puts it in there).  I randomly decided to try passing in PublishScriptFileName from the command line, and blamo, it worked!  I tried changing the <DeployScriptFileName> element in the .publish.xml file to <PublishScriptFileName>, but it still didn’t seem to have any effect.

So now if I wanted to deploy my database project to 3 separate databases, I could do so with the following code to first Build the project, and the Publish it to the 3 databases:

MSBuild /t:Build "[PathToBranch]\Database\Database.sqlproj"
MSBuild /t:Publish /p:TargetDatabaseName="[DbName1]";TargetConnectionString="Data Source=[Db.Server];Integrated Security=True;Pooling=False" /p:PublishScriptFileName="[DbName1].sql" /p:SqlPublishProfilePath="Template.publish.xml" "[PathToBranch]\Database\Database.sqlproj"
MSBuild /t:Publish /p:TargetDatabaseName="[DbName2]";TargetConnectionString="Data Source=[Db.Server];Integrated Security=True;Pooling=False" /p:PublishScriptFileName="[DbName2].sql" /p:SqlPublishProfilePath="Template.publish.xml" "[PathToBranch]\Database\Database.sqlproj"
MSBuild /t:Publish /p:TargetDatabaseName="[DbName3]";TargetConnectionString="Data Source=[Db.Server];Integrated Security=True;Pooling=False" /p:PublishScriptFileName="[DbName3].sql" /p:SqlPublishProfilePath="Template.publish.xml" "[PathToBranch]\Database\Database.sqlproj"

You could also instead just call MSBuild using the Build target with the PublishToDatabase parameter (which might actually be the safer bet); whatever you prefer.  I have found that once the database project is built once, as long as no changes are made to it then subsequent “builds” of the project only take a second or two since it detects that no changes have been made and skips doing the build.

If you have any questions or feedback, let me know.

Happy coding!

Parallel MSBuild FTW – Build faster in parallel

March 31st, 2012 1 comment

Hey everyone, I just discovered this great post yesterday that shows how to have msbuild build projects in parallel Smile

Basically all you need to do is pass the switches “/m:[NumOfCPUsToUse] /p:BuildInParallel=true” into MSBuild.

Example to use 4 cores/processes (If you just pass in “/m” it will use all CPU cores):

MSBuild /m:4 /p:BuildInParallel=true "C:devClient.sln"

Obviously this trick will only be useful on PCs with multi-core CPUs (which we should all have by now) and solutions with multiple projects; So there’s no point using it for solutions that only contain one project.  Also, testing shows that using multiple processes does not speed up Team Foundation Database deployments either in case you’re curious Winking smile

Also, I found that if I didn’t explicitly use “/p:BuildInParallel=true” I would get many build errors (even though the MSDN documentation says that it is true by default).

The poster boasts compile time improvements up to 59%, but the performance boost you see will vary depending on the solution and its project dependencies.  I tested with building a solution at my office, and here are my results (runs are in seconds):

# of Processes 1st Run 2nd Run 3rd Run Avg Performance
1 192 195 200 195.67 100%
2 155 156 156 155.67 79.56%
4 146 149 146 147.00 75.13%
8 136 136 138 136.67 69.85%


So I updated all of our build scripts to build using 2 cores (~20% speed boost), since that gives us the biggest bang for our buck on our solution without bogging down a machine, and developers may sometimes compile more than 1 solution at a time.  I’ve put the any-PC-safe batch script code at the bottom of this post.

The poster also has a follow-up post showing how to add a button and keyboard shortcut to the Visual Studio IDE to have VS build in parallel as well (so you don’t have to use a build script); if you do this make sure you use the .Net 4.0 MSBuild, not the 3.5 one that he shows in the screenshot.  While this did work for me, I found it left an MSBuild.exe process always hanging around afterwards for some reason, so watch out (batch file doesn’t have this problem though).  Also, you do get build output, but it may not be the same that you’re used to, and it doesn’t say “Build succeeded” in the status bar when completed, so I chose to not make this my default Visual Studio build option, but you may still want to.

Happy building!

:: Calculate how many Processes to use to do the build.
SET NumberOfProcessesToUseForBuild=1 
SET BuildInParallel=false
                SET NumberOfProcessesToUseForBuild=2
                SET BuildInParallel=true
MSBuild /maxcpucount:%NumberOfProcessesToUseForBuild% /p:BuildInParallel=%BuildInParallel% "C:\dev\Client.sln"

Setting up keyboard shortcuts to build solutions in MSBuild

June 1st, 2011 No comments

One of the greatest benefits of building your solution flies in MSBuild (vs in Visual Studio directly) is that it doesn’t lock up the Visual Studio UI, which can be a huge pain if you have a large solution that takes several minutes (or longer) to build.  Building your solution in MSBuild leaves you free to inspect code in Visual Studio while your solution is building.  The only problem is that to do this you have to open a command prompt and type the command + path every time to build.

If you want to be able to right-click on a solution file and build it in MSBuild from the Windows Explorer context menu, check out MSBuildShellExtension (it’s free).  Being able to build right from Windows Explorer (without having to even open Visual Studio) is cool and may be enough to passify you, but I wanted to be able to build my solution file at anytime from anywhere on my PC with a keyboard shortcut.

Below I outline how I’ve setup my system to build my solution files in MSBuild with a quick keystroke.  Setup only takes a few minutes and requires AutoHotkey to be installed (it’s free and awesome).

Step 1 – Install AutoHotkey.

Step 2 – Create a shortcut to the Visual Studio Command Prompt (2010), move it directly to the C: drive, and make sure it is called “Visual Studio Command Prompt (2010)” (it is referenced at this location with this name by the AutoHotkey script in the following steps, but can be changed if needed).



Step 3 – Create your AutoHotkey script……luckily, you don’t have to create it from scratch; you can use mine as your template and just adjust it to your liking . So copy and paste the script in the textbox below into a new text file, and then save it with the extension ".ahk", which is the AutoHotkey script extension (so you can just call it "AutoHotkeyScript.ahk" for example).  You will need to modify the code directory paths and solution file names in the script to match yours to build your solutions, but I’ve commented the script fairly thoroughly so it’s easy to see what it’s doing.

In my office we have both a Client solution and a Server solution, so I have the script setup to build the client solution with WindowsKey+C and the server solution with WindowsKey+S. We also work in multiple branches, so I have global variables near the top of the script that I can set to true to quickly switch between Development, QA, and Release branches.  I also have WindowsKey+U configured to open the code directory and WindowsKey+Q to open the database directory.  Obviously you can change the keyboard mappings to your liking; these are just the ones that I prefer.  As a side note here, just be aware that these will override the default windows key shortcuts; so in my case WindowsKey+U no longer opens up the Windows Ease of Access Center window.

; semicolon, such as this one, are comments.  They are not executed.
; This script has a special filename and path because it is automatically
; launched when you run the program directly.  Also, any text file whose
; name ends in .ahk is associated with the program, which means that it
; can be launched simply by double-clicking it.  You can have as many .ahk
; files as you want, located in any folder.  You can also run more than
; one ahk file simultaneously and each will get its own tray icon.
; Make it so only one instance of this script can run at a time (and reload the script if another instance of it tries to run)
#SingleInstance force
; Global Variables - Path settings, customization, etc.
; Set one of these to &quot;true&quot; to build from the Staging or Release branches, otherwise we'll use the development branch.
_UsingTFSStagingBranch := false
_UsingTFSReleaseCandidate := false
; Specify the Code Folder containing the Solution files to build
if (_UsingTFSReleaseCandidate == true)
    ; The directory of the current build's Code folder
    _CodeFolder := &quot;C:\dev\TFS\RQ4TeamProject\Release\RQ4\4.2.0\&quot;
else if (_UsingTFSStagingBranch == true)
    ; The directory of the current build's Code folder
    _CodeFolder := &quot;C:\dev\TFS\RQ4TeamProject\Staging\RQ4\&quot;
    ; The directory of the current build's Code folder
    _CodeFolder := &quot;C:\dev\TFS\RQ4TeamProject\Dev\RQ4\Core\&quot;
; Path to the database folder
_DatabaseFolder := &quot;C:\dev&quot;
; The path to the Visual Studio Command Prompt link
_VSCommandPromptPath := &quot;C:\Visual Studio Command Prompt (2010).lnk&quot;
_VSCommandPromptWindowName := &quot;Administrator: Visual Studio Command Prompt (2010)&quot;
; The position I want the MS Build window to move to when opened
_MSBuildWindowPositionX := 400
_MSBuildWindowPositionY := 270
; The MSBuild command to use
_MSBuildCommand := &quot;msbuild&quot; ; /verbosity:minimal&quot;
; WindowsKey+C - Build the Client.sln
#c UP::
; Make sure the keys have been released before continuing to avoid accidental commands
KeyWait LWin
;KeyWait c
;BuildSolution(_CodeFolder . &quot;RQ4.Client.sln&quot;)
; WindowsKey+S - Build the Server.sln
#s UP::
; Make sure the keys have been released before continuing to avoid accidental commands
KeyWait LWin
;KeyWait s
; WindowsKey+B - Build the Server.sln then Client.sln
#b UP::
; Make sure the keys have been released before continuing to avoid accidental commands
KeyWait LWin
;KeyWait b
; WindowsKey+U - Open the Code folder
#u UP::Run %_CodeFolder%
; WindowsKey+Q - Open the Database folder
#q UP::Run %_DatabaseFolder%
; Functions
    ; Let this function know that all variables except the passed in parameters are global variables.
    ; If the Visual Studio Command Prompt is already open
    if WinExist(_VSCommandPromptWindowName)
        ; Put it in focus
    ; Else the VS Command Prompt is not already open
        ; So open the Visual Studio 2008 Command Prompt
        Run %_VSCommandPromptPath%
        ; Make sure this window is in focus before sending commands
        WinWaitActive, %_VSCommandPromptWindowName%
        ; If the window wasn't opened for some reason
        if Not WinExist(_VSCommandPromptWindowName)
            ; Display an error message that the VS Command Prompt couldn't be opened
            MsgBox, There was a problem opening %_VSCommandPromptPath%
            ; Exit, returning failure
            return false
    ; Make sure this window is in focus before sending commands
    WinWaitActive, %_VSCommandPromptWindowName%
    ; Move the window to the position I like it to be
    WinMove, _MSBuildWindowPositionX, _MSBuildWindowPositionY
    ; Set it to the correct directory
    SendInput cd %_CodeFolder% {Enter}
    ;MsgBox %solutionPath%  ; Message box to display the Solution Path for debugging purposes
    ; Build the solution file
    SendInput %_MSBuildCommand% %solutionPath% {Enter}
    ; Return success
    return true

Step 4 – Have your AutoHotkey script automatically start when you login to Windows, so that you don’t have to manually launch it all the time.

Method 1:

This method is the easiest, but I discovered it after Method 2 (below).  Simply open up the Windows Start Menu, navigate to the Startup folder within All Programs, right-click on it and choose Open All Users.  Then simply paste a shortcut to your AutoHotkey script in this folder.  That’s it; the script will now launch whenever any user logs into Windows.  If you only want the script to run when you log into Windows (no other users), then just choose Open instead of Open All Users when right-clicking on the Startup folder.


Method 2:

Open the Windows Task Scheduler and create a new Basic Task.  Give it a name and description (something like “launch AutoHotkey script at login”), and then specify to have it run “When I log on”.  Then specify that you want it to “Start a program”, and then point it towards the AutoHotkey script you created in Step 3.  Before you finish the wizard, check off “Open the Properties dialog for this task when I click Finish”.  When that Properties dialog opens up, go to the Conditions tab and make sure none of the checkboxes under the Power category are checked off; this will ensure the script still launches if you are on a laptop and not plugged into AC power.  If you need your script to “Run as admin”, then on the General tab check off “Run with highest privileges”; this may be required for your script to perform certain actions the require admin privileges, so you can check it off just to be safe.

Open Task Scheduler(1)

 Create Basic Task in Task Scheduler


Basic Task Conditions

Run Scheduled Task as Admin_2

And that’s it.  Now you can build your solution file in MSBuild with a quick keystroke from anywhere on your computer.  I have chosen to use the Windows Key for my shortcut keys, but you don’t have to; you can use whatever keyboard shortcut you want.  And feel free to modify the script I provided to do whatever else you want; AutoHotkey is very powerful and can be used for so many things, so be sure to checkout their website for more examples of scripts and what it can do.  For example lots of people use it to automatically spell-correct as they type, or to automatically expand abbreviations (so I could type DS and hit tab, and have it expand to Daniel Schroeder, or type MyAddress and have it put my address in).

Happy Coding!

Solution won’t build on TFS Build Server

April 17th, 2011 1 comment

So if you are able to build (compile) the solution on your local machine, but it won’t build on the TFS build server and you are getting an error message similar to:

1 error(s), 1 warning(s)
$/TeamProject/Dev/RQ4/FBs/4.1.4_eSec/RQ4.Client.sln – 1 error(s), 1 warning(s), View Log File
IntegrationfrmGetIntegrationAuthorization.xaml.cs (1138): The type or namespace name ‘BillingFields’ could not be found (are you missing a using directive or an assembly reference?)
C:WindowsMicrosoft.NETFramework64v4.0.30319Microsoft.Common.targets (1360): Could not resolve this reference. Could not locate the assembly "IQ.Core.Resources, Version=, Culture=neutral, processorArchitecture=MSIL". Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.

Then the problem is likely one of the following:


1 – The project/dll being reference is set to "Specific Version", so open the reference’s properties and change it to "Any Version".


2 – The project is not set to be built under the specific configuration.  Right click on the solution in the Solution Explorer, and choose Configuration Manager.  All projects should be set to be build (i.e. have a checkmark), and all of their platforms should be set to the same value.  Change the Active Solution Platform to the platform you use and ensure that all projects are still set to always build.


3 – The path to the refenced project/dll is too long.  Windows/.NET has a limitation where the reference path cannot be more than 260 characters, and the directory it’s in cannot be more than 248 characters.  So the work around for this is usually to rename your build definition to something shorter, or if you just added a new project/namespace to the project, to shorten its name so the path stays under the limit.

Categories: Build, TFS Tags: , , , ,