Team City with Azure Virtual Machines

Posted by: Keith Patton | 12 February 2015

We recently moved our Team City build environment from on premise to the cloud, taking advantage of Azure Virtual Machines. This now allows for a more elastic build environment that powers up when we need it and is shut down when we don't.

It took us a few goes to get it right, so here’s a step by step guide so you don’t hit the same issues hopefully!

Azure VM Set Up

First off, you’ll need to create your VM(s)

  • Create a virtual network - This makes it easier to connect VMs and ensure the performance and security between them is optimized.
  • Create new Azure VM - We chose the D2 instance as it gave the right balance of bang for buck
  • Enable Port 80 for Team City - You can put it on any port you want, but this is all our VM is for so we run it on the default.
  • Attach all 4 Virtual Disks at maximum size of 1023GB each - Don't worry, you only pay for the space used, so assign them all now whilst you can.

Azure VM Configuration

Some basic server configuration is required, then important disk/storage config before configuration of your firewall. Finally you will install a database and Team City itself.

  • Install Feature .Net 2.0/3.5
  • Apply all updates and configure Windows Update to Download and Prompt and NOT to install automatically

Storage Pool Configuration

  • Create a Storage Pool called "TeamCity"
  • Assign 3 disks to a Virtual Disk called "TeamCity" with defaults
  • Assign a drive letter (we use T:) with name "TeamCity"
  • Create Storage Pool "MySQL"
  • Assign 1 desk to a Virtual Disk called "MySQL" with defaults"
  • Assign a drive letter "(we use M:) with name "MySQL"
  • This uses all 4 disks. You can the IOPS advantages by striping 3 of the disks but keep the MySQL and Team City activity separate on different disks which gives a good performance balance.

Windows Firewall Configuration

  • Add new rule to allow port 80
  • Call the rule "Team City"

MySql Installation

  • Custom install option
  • Configure Server Data Directory to M:\MySQL\MySQL Server 5.6\Data
  • Configure Log Paths to M:\MySQL\MySQL Server 5.6\Logs
  • Latest GA server and all connectors
  • All other file paths default (Core Install and Connectors to C:)

Team City Installation and Configuration

  • Install to C:\TeamCity
  • Install Data Directory: T:\TeamCityData
  • Configure Agent name to a friendly name and server url of your choosing
  • Configure Agent's System, Work and Temp Directories to D: (If you have chosen a D series VM for example this will give you a fast SSD for agent work!)
  • Browse to localhost to configure Team City
  • Confirm the Team City Data location (T:\TeamCityData)
  • Copy JDBC jar to T:\TeamCityData\lib\jdbc and refresh next screen
  • Create MySQL Database
    • cd C:\Program Files\MySQL\MySQL Server 5.6\bin
    • mysql -u root -p [password]
    • create database teamcity collate utf8_bin;
  • Enter database name as teamcity, username root and the root password
  • Click Proceed and Team City will start up
  • Install all the software you require to do builds (e.g. Visual Studio, Unity etc.)

Adding More Agent VMs

The initial Team City Azure VM you create will be default be both the Team City Server and its first Build Agent.

To create an additional team city build agent (which you can save), you go through exactly the same steps above to create an additional VM with the following changes:

Storage Pool Configuration

  • Stripe all 4 disks (as you do not have MySQL to install)

Windows Firewall Configuration

  • Add new rule to allow port 9090 (increment for each agent you create or just open 9090-909x where x is number of build agents)
  • Call the rule "Team City"

Installing Team City Build Agent

  • Browse to build server from the build agent VM and go to Agents > Install Build Agents to get msi

Build Agent Configuration

  • Open the buildAgent.properties to configure the build agent in Team City
  • ownPort - 9091 (increment from last port used for additional agents e.g. 909x)
  • name - set to name of the VM

SysPrep The Build Agent

Once you are done, sysprep /generalize via cmd line and this will ensure your VM can be used to easily create new Agent VMs (manually or automatically)

Team City Azure Plugin

Jetbrains have released a plugin to help with Azure/Team City integration.

When this is installed you will get a new “Cloud” tab on your Agents page in Team City. This will show all configured Azure build agents.

Within the configuration options you can create a new profile and add one or more vms or vm images. You can also specify when they should be shut down.

In our experience, it took too long to have it create the build agent from a VM image when our build queue got busy. Our preference therefore was to create a few real VMs (from a sysprepped build agent image as above) and then add them to the cloud profile on our main team city build server, as opposed to using the vm image.

Yes, you have a fixed number of servers in your pool, but the usage will scale up and down automatically. There is a standard limitation of 3 agents for a professional license of team city, so we didn’t want or need to go beyond this.

Agents will spin up automatically when the build queue requires them. It does take a few minutes for a VM to start up and the build agent to get connected, be patient here..

Task Scheduler to start/stop VMs

Azure VMs are not free ;)

In our case, we wanted the VMs to be active only when needed and so we create a simple “Start VMs” and “Stop VMs” task scheduler behaviour to star the VMs at 8am each week day and shut down at midnight Tue-Sat. This task scheduler can be hosted on a very small VM in Azure or on a local server as required. This, in addition to the team city azure plugin behavior of shutting down idle build agents means we paying a fraction of what we would be if the VMs were on all the time.

References

These posts were invaluable in learning about team city, azure and the cloud plugin developed recently http://blog.maartenballiauw.be/post/2013/08/05/An-autoscaling-build-farm-using-TeamCity-and-Windows-Azure.aspx http://azure.microsoft.com/blog/2014/11/13/running-builds-on-azure-with-teamcity-and-the-azure-sdk-for-java/

Want to get started?

We would love to help with your next app or game, please do get in touch.