Using Static Methods from the .NET Framework in MSBuild – a List of All Property Functions

by Oliver 9. June 2016 20:20

In the MSBuild deployment script for our discoverize portals we use a number of useful functions from the .NET framework, e.g.: $([System.IO.File]::Exists($file)) $([System.IO.Path]::GetFileName($(Destination))) $([System.IO.Directory]::GetDirectories("$(Folder)")) $([System.DateTime]::Now.ToString($(TimestampFormat))) These methods are called Property Functions and have been made available for use in MSBuild scripts since version 4. Here's the full list of .NET framework types whose static methods or properties you can use almost anywhere in you MSBuild scripts: System.Byte System.Char System.Convert System.DateTime System.Decimal System.Double System.Enum System.Guid System.Int16 System.Int32 System.Int64 System.IO.Path System.Math System.UInt16 System.UInt32 System.UInt64 System.SByte System.Single System.String System.StringComparer System.TimeSpan System.Text.RegularExpressions.Regex Microsoft.Build.Utilities.ToolLocationHelper There are a few rather useful methods from some more types that you can also use: System.Environment::CommandLine System.Environment::ExpandEnvironmentVariables System.Environment::GetEnvironmentVariable System.Environment::GetEnvironmentVariables System.Environment::GetFolderPath System.Environment::GetLogicalDrives System.IO.Directory::GetDirectories System.IO.Directory::GetFiles System.IO.Directory::GetLastAccessTime System.IO.Directory::GetLastWriteTime System.IO.Directory::GetParent System.IO.File::Exists System.IO.File::GetCreationTime System.IO.File::GetAttributes System.IO.File::GetLastAccessTime System.IO.File::GetLastWriteTime System.IO.File::ReadAllText The general pattern to call a property method is: $([Class]::Method(Parameters)). Beyond the above mentioned methods, MSBuild offers some more helpful ones that are invoked on the MSBuild pseudo class: [MSBuild]::DoesTaskHostExist(string runtime, string arch) [MSBuild]::GetDirectoryNameOfFileAbove(string p, string f) [MSBuild]::GetRegistryValue(...) [MSBuild]::GetRegistryValueFromView(...) [MSBuild]::MakeRelative(string path1, string path2) [MSBuild]::ValueOrDefault(string value, string default) [MSBuild]::Escape(string unescaped) [MSBuild]::Unescape(string escaped) [MSBuild]::Add(double a, double b) [MSBuild]::Add(long a, long b) [MSBuild]::Subtract(double a, double b) [MSBuild]::Subtract(long a, long b) [MSBuild]::Multiply(double a, double b) [MSBuild]::Multiply(long a, long b) [MSBuild]::Divide(double a, double b) [MSBuild]::Divide(long a, long b) [MSBuild]::Modulo(double a, double b) [MSBuild]::Modulo(long a, long b) [MSBuild]::BitwiseOr(int first, int second) [MSBuild]::BitwiseAnd(int first, int second) [MSBuild]::BitwiseXor(int first, int second) [MSBuild]::BitwiseNot(int first) The smart thing about those arithmetic methods is that MSBuild converts string values to a matching number type on the fly, so there's no need for any explicit type conversion. And that would be the whole spectrum of property functions in MSBuild. For further reading, please turn to the official documentation on the MSDN. Happy coding! photo credit: Paddington Reservior Gardens Roof via photopin (license)

Update (or Delete) an entry from appSettings in web.config using MSBuild and the XmlUpdate task

by Oliver 11. May 2016 12:24

How do I modify an entry in the appSettings node during deployment? We use a custom MSBuild script to deploy our discoverize portals and wanted to set a portal dependent key, namely NewRelic.AppName. So how did we do it? We use the XmlUpdate task from the MSBuild Community Tasks project. Here's the code: <!-- Import all MSBuild Community Tasks --> <Import Project="$(LibFolder)\msbuild\MSBuild.Community.Tasks.Targets" />   <!-- Define the property value we want to use below --> <PropertyGroup>   <NewRelicAppName>$([System.IO.Path]::GetFileName($(Destination))), Discoverize Portals</NewRelicAppName> </PropertyGroup>   <!-- Update NewRelic.AppName key in appSettings of web.config --> <XmlUpdate XmlFileName="$(Destination)\web.config"   XPath="/configuration/appSettings/add[@key='NewRelic.AppName']/@value"   Value="$(NewRelicAppName)" /> I'm no XPath guru, so I was glad to find this thread that contained the XPath expression needed to select a certain appSettings entry by key and update its value. If you seek to delete an element from an XML file, this stackoverflow answer taught me that there is also a Delete attribute on the XmlUpdate task :-) Happy deploying!

ASP.NET vNEXT, Docker, and the Future of Application Development and Deployment

by Oliver 3. November 2014 09:16

It's been an impressive year so far in the realms of software development and deployment, especially with ASP.NET vNEXT enabling per-application bundling of not only the .NET runtime but even the CLR needed for your app Docker standardizing the software delivery process by use of Linux containers (runs on Windows in a VM), (here's A Docker ‘Hello World' With Mono) and now Microsoft announcing native Docker Support for Windows Server Now, it took me a while to understand that we're witnesses of nothing less than a revolution in software development. The Vision: Build Your App Anywhere, Bundle It, and Run It Anywhere (Else) The clouds have been with us for a couple of years now and have started to provide real benefit beyond "moving your stuff to somewhere else". What's emerging now, with Docker and also the new ASP.NET runtime bundling, is something completely new: Application Containers. They don't have either specific OS requirements – Docker will be supported natively on Windows Server soon, ASP.NET runs on Linux today – nor need they a specific technology stack installed on the target machine (as with PaaS) because they bring all of the necessary runtime along. But they're also not large VMs bundled with your application, which carry a significant maintenance overhead (when using IaaS). Virtualized application containers are the sweet spot between IaaS and PaaS. Go ahead and read that post – it's eye-opening.

Productivity boost with MSBuild: use /maxcpucount

by Oliver 28. January 2014 21:24

This is embarrassing. For the n-th time during the past couple of years I've felt an unease waiting for our projects (read: solutions) to compile. I kept seeing this: This is MSBuild using 1 (!), yes, one!, of the 8 CPU cores I've sitting in my machine to get my work done. What about the other 7? Why don't you use them, MSBuild? With that single core, currently my simple local build of our project discoverize takes around 36 seconds: Tell MSBuild to use all cpu cores Well, it's as easy as adding /m or /maxcpucount to your msbuild command line build to boost your build times:   Down to 8 seconds with 3 additional characters: [space]/m. That's easily a 4.5 times improvement! Your mileage may vary Of course, every project is different, so your speed increase might be higher or a lot lower than what I've seen. But it's an easy measure to get at least some improvement in build times with very little effort. Don't trust Visual Studio on that one, though – the solution builds slowly there, still. For reference, let me tell you, that the /maxcpucount switch can actually take a parameter value like so: /maxcpucount:4. So if you lots of other stuff going on in the background or I don't know for what reason, really, you can limit the number of cpus used by MSBuild. Props to the Orchard team for a highly parallelizable build One of the specifics of the Orchard source code that's the base for discoverize is the very loose coupling between the 70+ projects in the solution. This allows MSBuild to distribute the compilation work to a high number of threads because there are almost no dependencies between the projects that MSBuild would have to respect. Great work! Happy building!

Test Driving AppHarbor – A Walkthrough and Review

by Oliver 15. March 2013 20:50

For some time now, I've wanted to check out AppHarbor, a cloud service to host .NET applications that includes a build environment, executes tests and deploys successful builds to one or more app servers. They use Amazon's cloud computing infrastructure as their backend. The smallest package is free so there's no good reason not to check it out. Getting my first application up and running First, you need to Sign up, confirm the link in the confirmation email, and log in. This part took about 2 minutes. (Created a new KeePass entry with an uncrackable password on the way.) Then, create an application, entering a name and the geographical region you want your application to be hosted at: Once you're done with that, you can choose where your code is hosted – this assumes you version control your source code using e.g. BitBucket, CodePlex, or Git. They also have a solution for the situation where you don't host your code anywhere, using a built-in Git repository. I didn't use that option, though, since I have an account at GitHub. Clicking on "Configure GitHub to deploy to AppHarbor" directs you to the GitHub logon screen (if you're logged out) where you simply sign in. Now, the following dialog was a bit spooky: What I read between the lines is something like: all your base are belong to us! I mean, it basically says that they can do to all of my projects … well, anything, really. Since I didn't want to create a new account just to try out AppHarbor and, honestly, because I somehow felt that they wouldn't destroy all of my work, I clicked "Authorize app". Phew! Remark: You might choose to use a different GitHub account for your deployments, using e.g. copies of your repositories locally where you just copy everything you need from your dev repo. Then you can grand AppHarbor access to that account without much ado. Now, the AppHarbor app took over and I chose a repository for the application I created earlier: Once I chose a repo for my first app, I got to see the first Build status message – here AppHarbor is building my app for the first time: A few seconds after AppHarbor was done building, and testing, and deploying my app after I had clicked the Deploy button, my app was ACTIVE :-) Under the Hostnames link I found that they had given it http://mathie.apphb.com/ and after a couple of seconds I saw my app online on AppHarbor. That was easy – how about deploying a new version? I'm quite surprised at how easy (and fast!) it was to get my first app up and running. For completeness, I wanted to check how AppHarbor would handle my pushing some changes to the master branch of my repo. Here we go: The commit was picked up within seconds! Another click on Deploy gets the new version out there. Remark: On their homepage they say that apps get automatically deployed once build and test runner finish successfully. This was not the case here, and I didn't find any setting to enable this. If you know how that works, please leave a comment. Update: Looks like this works out of the box, you just need to wait a minute or two for their deployment agent to pick up the new version. I updated my app just now and it got deployed by itself :-) Go back in time – it's easy, too Now, this is a nice feature: you can deploy any version of your application with a click of a button! So, if for some reason, you discover that the new version has some flaw, go back to an older one: What else do they offer? AppHarbor contains an add-on infrastructure and already offers several add-ons that you can install with your application. Most of them charge an extra monthly fee, some of them also offer a free plan. There are mostly analytics add-ons and DB engines, including dedicated MS SQL Server, RavenDB, MySQL, a PostgrSQL flavor, and a few more. Interesting platform with big ease of setup and a free plan Should be good for any smallish app that you just want to set up and forget about! For $10/month you can also assign your own host names which makes this a viable solution. It saves you from installing and maintaining a separate build server with something like TeamCity or CruiseControl running and is also almost easier to set up. Where are you gonna host your next app?

Test Driving AppHarbor – A Walkthrough and Review

by Oliver 15. March 2013 20:50

For some time now, I've wanted to check out AppHarbor, a cloud service to host .NET applications that includes a build environment, executes tests and deploys successful builds to one or more app servers. They use Amazon's cloud computing infrastructure as their backend. The smallest package is free so there's no good reason not to check it out. Getting my first application up and running First, you need to Sign up, confirm the link in the confirmation email, and log in. This part took about 2 minutes. (Created a new KeePass entry with an uncrackable password on the way.) Then, create an application, entering a name and the geographical region you want your application to be hosted at: Once you're done with that, you can choose where your code is hosted – this assumes you version control your source code using e.g. BitBucket, CodePlex, or Git. They also have a solution for the situation where you don't host your code anywhere, using a built-in Git repository. I didn't use that option, though, since I have an account at GitHub. Clicking on "Configure GitHub to deploy to AppHarbor" directs you to the GitHub logon screen (if you're logged out) where you simply sign in. Now, the following dialog was a bit spooky: What I read between the lines is something like: all your base are belong to us! I mean, it basically says that they can do to all of my projects … well, anything, really. Since I didn't want to create a new account just to try out AppHarbor and, honestly, because I somehow felt that they wouldn't destroy all of my work, I clicked "Authorize app". Phew! Remark: You might choose to use a different GitHub account for your deployments, using e.g. copies of your repositories locally where you just copy everything you need from your dev repo. Then you can grand AppHarbor access to that account without much ado. Now, the AppHarbor app took over and I chose a repository for the application I created earlier: Once I chose a repo for my first app, I got to see the first Build status message – here AppHarbor is building my app for the first time: A few seconds after AppHarbor was done building, and testing, and deploying my app after I had clicked the Deploy button, my app was ACTIVE :-) Under the Hostnames link I found that they had given it http://mathie.apphb.com/ and after a couple of seconds I saw my app online on AppHarbor. That was easy – how about deploying a new version? I'm quite surprised at how easy (and fast!) it was to get my first app up and running. For completeness, I wanted to check how AppHarbor would handle my pushing some changes to the master branch of my repo. Here we go: The commit was picked up within seconds! Another click on Deploy gets the new version out there. Remark: On their homepage they say that apps get automatically deployed once build and test runner finish successfully. This was not the case here, and I didn't find any setting to enable this. If you know how that works, please leave a comment. Update: Looks like this works out of the box, you just need to wait a minute or two for their deployment agent to pick up the new version. I updated my app just now and it got deployed by itself :-) Go back in time – it's easy, too Now, this is a nice feature: you can deploy any version of your application with a click of a button! So, if for some reason, you discover that the new version has some flaw, go back to an older one: What else do they offer? AppHarbor contains an add-on infrastructure and already offers several add-ons that you can install with your application. Most of them charge an extra monthly fee, some of them also offer a free plan. There are mostly analytics add-ons and DB engines, including dedicated MS SQL Server, RavenDB, MySQL, a PostgrSQL flavor, and a few more. Interesting platform with big ease of setup and a free plan Should be good for any smallish app that you just want to set up and forget about! For $10/month you can also assign your own host names which makes this a viable solution. It saves you from installing and maintaining a separate build server with something like TeamCity or CruiseControl running and is also almost easier to set up. Where are you gonna host your next app?

Automatic Deployment of a New Discoverize Portal

by Oliver 11. March 2013 15:02

This is a technical post about how we automatically deploy new portals that will be run by our own portal software discoverize. After filling in and submitting our portal creation form we do the following: Save the entered information into /App_Data/<PortalName>/portal.xml Trigger the TeamCity build through a web request to a specific URL Send an email with the request status or any errors that occurred Display a Thank You page. Now, the TeamCity build configuration that was triggered under 2. does the following: Execute the Powershell script create-portal.ps1 in c:\projects\discoverize create the AppPool Discoverize-Portals if it doesn't exist yet (all our portals run within it) iterate over all folders inside wwwroot\discoverize.com\App_Data if a folder with the same name exists under wwwroot\discoverize-portals, skip it else move the current folder from wwwroot\discoverize.com\App_Data to wwwroot\discoverize-portals add a line with the full path to the new portal folder to newportals.txt (e.g. C:\inetpub\wwwroot\discoverize-portals\<PortalName>) add a new site to IIS and set its AppPool to Discoverize-Portals start the new site in IIS (needs Admin privileges, that's why we use a parameterized task scheduler task for that) Build the source code from the repository Execute the Deploy-NewPortals target from deploy.proj using MSBuild in c:\projects\discoverize Iterate over the entries in newportals.txt, for each entry do: Copy the build output from step 2. to wwwroot\discoverize-portals\<PortalName> Run the site setup (this uses Orchard's setup method) with our own recipe (using Orchard.exe) Call an action (SetupComplete) on the new site to mark deployment as done Create a default entry so there's something to look at (using Orchard.exe) The SetupComplete action then finishes doing this: save the information of the portal.xml into an application specific configuration file create new user "portal-owner" for content management send an email about the successful installation of the new portal Quite complex, if I look at it now, but it works!

MSBuild and Visual Studio 2010 generate different DLLs

by Oliver 6. September 2011 22:18

Recently, we encountered a quite surprising behavior of MSBuild – the continuous integration build of our new collaborative Todo Management app (we hope to go into beta soon!) would produce a broken version whereas the local build with VS2010 was all smooth and well. Our admin and tester already posted about this problem over at his blog: MSBuild does not build like Visual Studio 2010. The exception message finally led me down the right path: Server Error in '/' Application. No constructors on type 'Teamaton.TodoCore.Repositories.TodoRepositoryJson' can be found with 'Public binding flags'. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: Autofac.Core.DependencyResolutionException: No constructors on type 'Teamaton.TodoCore.Repositories.TodoRepositoryJson' can be found with 'Public binding flags'. The TodoRepositoryJson is a type we used at the very beginning of our development to quickly get started using a JSON document as data store. Later we switched to SQLite, so now we have another implementation: TodoRepositoryDb. Both implement the same interface ITodoRepository. Turns out, the Autofac type registering code was the culprit: 1: var builder = new ContainerBuilder(); 2: builder.RegisterAssemblyTypes(typeof (Todo).Assembly) 3: .Where(t => t.Name.Contains("Repository")) 4: .AsImplementedInterfaces() 5: .InstancePerLifetimeScope(); What worked with Visual Studio, didn’t work with MSBuild: obviously – well, now it is – both ITodoRepository implementations were registered with Autofac, and while Autofac’s assembly scanning delivered them in the order we assumed from the DLL built with VS  – first, TodoRepositoryJson, and second, TodoRepositoryDb, thus overriding the first registration – MSBuild seems to build a DLL which returns the inverse order! Very strange. Honestly, I’m not familiar with the anatomy of DLLs and surprised by this result. But it’s the only explanation I’ve found so far. Well, the solution to the problem is, of course, to care more about what we register with Autofac and in which order. Happy coding, Oliver

The Teamaton tool belt: a collection of small but useful tools for every day development and administration

by Oliver 26. January 2011 20:30

Today: a simple hosts file editor Today I set up a new project on GitHub: the Teamaton tool belt! It shall serve us as a central store for small tools, probably mostly command-line, built for a single purpose. The first tool in our new tool belt is: HostsEditor. I wrote this small command line utility after reading this blog post: http://apdubey.blogspot.com/2008/09/edit-host-file-by-batch-file.html. We were looking for an easy way to edit the Windows hosts file from a script for a larger development environment setup that we wanted to fully automate including IIS website, bindings and new entries in the said hosts file. Of course, for a one-timer echo does the job, but don’t try to run the setup script twice on the same maching – you’ll get duplicate entries very quickly. Reading about how other people are asking the author of the above post how to avoid the duplicates or how to erase entries from the hosts file, I set out to simply write such a utility myself and share it with the rest of the world. If there is one other person out there to whom it will be useful I will be happy. So, HostsEditor does the following: Adds and removes entries to and from the Windows hosts file. Automatically backs up the original hosts file to hosts.bck. Does not add duplicate entries; instead prints a warning. For usage in automation scripts there is /q switch to suppress all info messages. That’s all there is to it for now. But there’s more to come! Stay tuned, Oliver

.NET Build Performance optimieren!

by robert 12. December 2009 00:45

In den letzten Wochen hatte ich die die Möglichkeit Strategien für Beschleunigung von Builds mit Visual-Studio 2008 und MsBuild zu untersuchen. Hier meine Ergebnisse. 1:) Multiprozessor Build Visual Studio unterstützt beim Build per Default nur eine Thread. Doch es geht mehr! Das Zauberwort heißt: "msbuild /m". Detailiert wird das Problem und die Lösung von Scott Hanselman beschrieben: Parallel Builds and Multicore CPUs Parallel MSBuilds from within the Visual Studio IDE 2:) Einzelnes Output Directory und keine "lokalen Kopien". Per Default werden Build Ergebnisse in /projekt/bin/<Target>/ kopiert. Alle abhängigen Assemblies werden per Default ebenso kopiert, also auch Build-Ergebnisse anderer Projekte. Nehmen wir eine Beispiel-Solution mit 50 Projekten. Jedes Projekt hat ungefähr 10.000 Zeilen Code und nehmen wir an die 50 Projekte sind gerechtfertigt. Wenn jedes Projekt mehrfach von anderen Projekten referenziert wird und 10.000 Zeilen ohne .resx Dateien eine DLL Größe von ca.: 250kb ergeben und eine PDB Größe von ca.:500kb, dann ergeben sich, wenn jedes Projekt ungefähr 10x referenziert wird , 10x750kb*50 - also 375 MB zu kopierende Daten. Obwohl eigentlich nur 37MB kopiert werden müssten. Die Lösung für das Problem? Das Build in ein einzelnes Verzeichnis durchführen, ohne die Abhängigen immer wieder zu kopieren. Hierfür muss neben der Pfadangabe für das OutputDirectory folgende Einstellung vorgenommen werden: 3:) Abhängigkeiten minimieren Wenn I/O Zeiten (siehe 2) kein Problem sind und alle verfügbaren Kerne ausgelastet sind (siehe 1), dann ist der größte verbleibende Zeitfaktor das Auflösen von Abhängigkeiten. Eine Analyse lässt sich mit dem MSBUILD-Profiler durchführen.  Lösung des Problems? Weniger Abhängigkeiten und Projekte zusammenfassen :-) 4:) .licx Dateien nicht Teil des Projekts LC.exe ist langsam. Es lässt sich Build Zeit sparen, wenn Lizenz-Informationen nur bei Bedarf in die Assembly "gebaut" werden. .licxs Dateien sollten nicht eingecheckt werden und nicht Teil des Dev-Builds sein. Was noch? Solutions entkoppeln und Modular arbeiten? Klar :-) Ein schnelle Entwicklungsmaschine hilft, eine gute SSD Platte und ein schneller Prozessor helfen. Wenn man damit nicht mehr weiterkommt, dann gilt es die Solution zu zerlegen. Als Richtwert sollte ein Clean-Build von 500.000 Zeilen Code (Nettozeilen ohne Kommentare) unter einer Minute liegen.

enjoyed the post?

Tags:

Build

About Oliver

shades-of-orange.com code blog logo I build web applications using ASP.NET and have a passion for javascript. Enjoy MVC 4 and Orchard CMS, and I do TDD whenever I can. I like clean code. Love to spend time with my wife and our children. My profile on Stack Exchange, a network of free, community-driven Q&A sites

About Anton

shades-of-orange.com code blog logo I'm a software developer at teamaton. I code in C# and work with MVC, Orchard, SpecFlow, Coypu and NHibernate. I enjoy beach volleyball, board games and Coke.