Self-hosting an Azure DevOps agent in WSL2.

The thesis.

Self-hosting a build agent on hardware you already own is one of the higher-yield, lower-risk capital allocations available to a working developer. Capex is zero if you have a machine. Opex is single-digit dollars a month in electricity, plus $15 per concurrent slot past the first if you're on Azure DevOps Services. On Azure DevOps Server, opex is the electricity. That's it.

What you buy is a step-function improvement in build latency — typically three to ten times faster than Microsoft-hosted for a .NET or Node workload — driven almost entirely by warm caches and modern hardware, not by anything clever. The cold-start tax disappears: NuGet packages stay restored, Docker layers stay cached, the dotnet SDK stays installed, the agent stays paged in. The cost of a build approaches the cost of the work it's actually doing.

The risk is small and mostly operational. You own patching, hardware failure, and the chance that your home internet drops while a deploy is mid-flight. None of these are catastrophic if you keep Microsoft-hosted in your back pocket as a fallback and don't put a single self-hosted agent in the critical path of a production change that requires nine-nines availability.

The licensing math.

Three pricing models matter and they're not interchangeable.

The pricing is per concurrent job slot, not per agent and not per machine. You can register thirty agents and still only run one job at a time on the free tier. Conversely, four agents on a single VM count as four slots if all four are busy. The slot is the unit of work; the agent is the worker that consumes it. Confuse these and you'll overbuy.

The free Microsoft-hosted parallel evaporates the moment you add a paid user to the organization. Teams upgrade a couple of Stakeholders to Basic and discover their CI queues silently. The self-hosted free slot is unaffected. And on ADO Server — the on-prem product, formerly TFS — the parallel-job line item doesn't exist at all. You pay for users and CALs, not concurrency.

The Windows detour.

The first wrong turn I took was assuming that because the build machine ran Windows, the agent should too. The reasoning seemed sound at the time: Windows host, Windows tooling, Windows-native .NET SDK installed via winget, agent service running as NT AUTHORITY\NetworkService. A clean configuration.

What broke it was that the pipelines were written for Linux. The chitra.art build chain had been running on Microsoft-hosted Ubuntu images for years. The Bash@3 tasks reached for bash.exe on the Windows agent, found C:\Windows\System32\bash.exe — which is the legacy WSL launcher, looking for a distro registered to NetworkService, which has none — and exited with the helpful message:

Windows Subsystem for Linux has no installed distributions.

Getting Git Bash on the system PATH meant discovering that the winget install of Git for Windows had added C:\Program Files\Git\cmd (containing git.exe) to PATH but not C:\Program Files\Git\bin (containing bash.exe). Fixed that. The next step then failed:

grep: F:agent_work1s/frontend/.npmrc: No such file or directory

That gibberish path is what you get when an ADO macro like $(Build.SourcesDirectory) substitutes to F:\agent\_work\1\s and bash then eats every backslash as an escape character. \a, \_, \1, \s — all dutifully consumed, leaving the letters behind. The fix is $BUILD_SOURCESDIRECTORY plus cygpath -u. Workable. But the realization sitting behind that fix was that I'd be paying this tax on every bash script in every pipeline forever, plus path-tool differences, plus line-endings, plus whatever else the impedance mismatch would surface next quarter.

The Windows agent was the wrong host for these pipelines. Not because Windows is bad, not because the agent doesn't work — both are fine — but because matching the agent OS to the pipeline target OS is the single decision that eliminates the most subsequent friction. I had inverted it.

The right architecture.

The answer, in retrospect, was sitting on the same physical machine: install the agent inside WSL2 Ubuntu. Same hardware, same Docker daemon (now reachable over a Unix socket, no TCP bridge required), same fast NVMe. Linux filesystem, Linux paths, Linux toolchain. The pipelines stop fighting the host and just run.

The stack, with the wrong alternatives flagged:

• Windows 11 as the host OS — already there, didn't have to change it.
• WSL2 with Ubuntu 24.04, distro stored on the F: drive at F:\wsl\ubuntu\ext4.vhdx.
• Docker Engine via apt, not Docker Desktop. The engine itself is Apache 2.0 and free regardless of company size. Desktop has a commercial licensing trap and only adds a GUI — useless on a build box.
• The agent in ~/agent inside Ubuntu, running as a systemd service. Not /mnt/f/agent. More on that in a moment.

The agent lives in the WSL filesystem, not on /mnt/f. This was non-obvious. The instinct is to put _work on the fast NVMe by referencing it as /mnt/f/agent/_work. Don't. /mnt/f is a bridge to F: via the 9P protocol, and 9P adds per-syscall latency that destroys small-file workloads — which is most of what CI does. Native ext4 inside the WSL vhdx hits full NVMe speed because the vhdx file itself is on F:. Same hardware. Different path. About 5–10× faster on npm install or dotnet restore.

Setup, condensed.

The setup with the missteps removed. Reproduce in order.

From an admin PowerShell, confirm virtualization and install WSL:

systeminfo | Select-String "Hyper-V|Virtualization"

wsl --install
# Reboot when prompted

# Move the distro to F: so the vhdx lives on the fast drive
wsl --shutdown
mkdir F:\wsl
wsl --export Ubuntu F:\wsl\ubuntu-backup.tar
wsl --unregister Ubuntu
mkdir F:\wsl\ubuntu
wsl --import Ubuntu F:\wsl\ubuntu F:\wsl\ubuntu-backup.tar --version 2
del F:\wsl\ubuntu-backup.tar

ubuntu config --default-user yourname

Inside Ubuntu, enable systemd, then install Docker Engine:

sudo tee /etc/wsl.conf <<EOF
[boot]
systemd=true

[user]
default=$USER
EOF

# wsl --shutdown from PowerShell, then back in:

sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io \
  docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
sudo systemctl enable --now docker
docker run --rm hello-world

Create a PAT in Azure DevOps scoped to Agent Pools (read, manage). Create a pool named home-linux. Then, still inside Ubuntu, register the agent — in ~/agent, on the ext4 side:

mkdir -p ~/agent && cd ~/agent

AGENT_VER="TODO: fill in current version from github.com/microsoft/azure-pipelines-agent/releases"
curl -O https://vstsagentpackage.azureedge.net/agent/$AGENT_VER/vsts-agent-linux-x64-$AGENT_VER.tar.gz
tar zxf vsts-agent-linux-x64-$AGENT_VER.tar.gz
rm vsts-agent-linux-x64-$AGENT_VER.tar.gz

sudo ./bin/installdependencies.sh

./config.sh --unattended \
  --url https://<your-ado-instance> \
  --auth pat --token <PAT> \
  --pool home-linux \
  --agent "$(hostname)-wsl" \
  --work _work \
  --acceptTeeEula

sudo ./svc.sh install $USER
sudo ./svc.sh start
sudo ./svc.sh status

If config.sh returns "needs Manage permissions for pool," your PAT-issuing user doesn't have Administrator on the pool. Grant it via Collection Settings → Agent pools → Security. The PAT carries the user's permissions, not its own.

Install the toolchain and restart so it advertises capabilities:

# .NET
wget https://dot.net/v1/dotnet-install.sh && chmod +x dotnet-install.sh
sudo ./dotnet-install.sh --channel 10.0 --install-dir /usr/share/dotnet
sudo ln -sf /usr/share/dotnet/dotnet /usr/local/bin/dotnet

# Node
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs

# Common pipeline utilities
sudo apt-get install -y jq git zip unzip rsync build-essential default-jdk

# Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

sudo systemctl restart "vsts.agent.*"

Then point the pipeline at the new pool:

pool:
  name: home-linux

Gotchas worth knowing.

None of these are showstoppers. All of them have eaten an evening from somebody.

WSL doesn't auto-start without a user session.

WSL2 is bound to a user session by default. After a Windows reboot, WSL doesn't come up until something triggers it — a login, an explicit wsl command, a scheduled task. If your agent is in WSL and Windows reboots overnight, the agent is dead until you log in. Workable answers: enable auto-login on a dedicated build box, set up a startup scheduled task that wakes WSL, or just accept the occasional manual nudge. For a daily-driver workstation, the last one is fine.

Docker Desktop leftovers.

If Docker Desktop was ever installed on the machine, it leaves behind a desktop-linux Docker context that overrides DOCKER_HOST. Symptom: docker version shows Context: desktop-linux and tries to connect to a named pipe that no longer exists. Fix:

docker context use default
docker context rm desktop-linux

It will also have registered docker-desktop and possibly docker-desktop-data WSL distros. Unregister both if you're not using them — they consume memory and create confusion about which distro wsl drops you into.

The vhdx-on-F: trap.

"Why isn't my agent on F:, it's my fastest drive" was a question I asked myself before learning that the vhdx is on F:. Native Linux filesystem operations inside Ubuntu hit ext4-on-vhdx-on-F: at full NVMe speed. /mnt/f bridges to the same physical drive through 9P and is dramatically slower for small-file metadata operations — which is most of what CI does.

The drive is the same. The route to it is the difference. Keep ~/agent in the Linux home directory.

Speed, in tiers.

The performance work, ordered by effort-to-impact. Most teams stop somewhere in Tier 2 and that's enough.

Tier 1 — Free, ten minutes.

Shallow clone, skip the warm-cache nuke, and chain --no-restore / --no-build through the .NET steps:

- checkout: self
  fetchDepth: 1
  fetchTags: false
  lfs: false
  submodules: false

workspace:
  clean: outputs   # NOT 'all' — that nukes warm caches

variables:
  DOTNET_CLI_TELEMETRY_OPTOUT: 1
  DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
  DOTNET_NOLOGO: 1
  NUGET_XMLDOC_MODE: skip

dotnet restore Solution.sln
dotnet build Solution.sln --no-restore -c Release --nologo
dotnet test Solution.sln --no-restore --no-build -c Release --nologo
dotnet publish src/Api/Api.csproj --no-restore --no-build -c Release --nologo

Each skipped restore saves 20–60 seconds. Shallow clone saves another half-minute to several minutes on a long-history repo. clean: outputs over clean: all preserves node_modules, the NuGet cache, and Docker layers — easily worth more than every other Tier 1 change combined.

Tier 2 — Cache@2, the standard pattern.

Belt-and-suspenders alongside the persistent self-hosted home directory. Lock files matter — without committed packages.lock.json the cache key is unstable and you'll miss on every minor SDK fluctuation.

- task: Cache@2
  displayName: NuGet cache
  inputs:
    key: 'nuget | "$(Agent.OS)" | **/packages.lock.json'
    restoreKeys: |
      nuget | "$(Agent.OS)"
    path: $(NUGET_PACKAGES)

variables:
  NUGET_PACKAGES: $(Pipeline.Workspace)/.nuget/packages

Tier 3 — Docker layer caching.

Buildx with a persistent local cache is the single biggest win for container-heavy pipelines. Equally important is Dockerfile layer ordering: COPY the .csproj and lock files first, restore, then copy source. A code change should not invalidate the package-restore layer. If yours does, your incremental builds are paying the full first-build cost on every run.

- script: |
    docker buildx create --use --name builder 2>/dev/null || true
    docker buildx build \
      --cache-from type=local,src=/tmp/buildx-cache \
      --cache-to type=local,dest=/tmp/buildx-cache-new,mode=max \
      --load -t chitra-api:$(Build.BuildId) \
      -f src/Api/Dockerfile .
    rm -rf /tmp/buildx-cache
    mv /tmp/buildx-cache-new /tmp/buildx-cache

Tier 4 — Local package proxies.

BaGet for NuGet, Verdaccio for npm, a plain registry:2 for Docker — all on the same WSL host. A small docker-compose file, an evening of configuration, and your restore steps stop crossing the public internet for packages you've already pulled. The first restore populates; every subsequent one hits the LAN. On a steady-state pipeline dotnet restore goes from minutes to seconds.

Tier 5 — Parallelism and pipeline hygiene.

Independent jobs in parallel via dependsOn: []. Test parallelism via xUnit's collection settings. Path filters on triggers to skip rebuilds on docs-only changes. condition: ne(variables['Build.Reason'], 'PullRequest') on expensive optional work. None of these are individually transformative; collectively they trim the long tail. Before optimizing further, profile — the run summary lets you sort steps by duration, and usually 60% of build time is in two or three steps. Fix those, leave the rest.

What would change my mind.

Three scenarios would push me back to Microsoft-hosted, in whole or in part.

A team that scales past five concurrent developers. The operational overhead of self-hosting — patching, hardware replacement, the inevitable "agent is offline" Slack message — eventually competes with paying Microsoft to handle it. The cross-over depends on how good your ops culture is, but somewhere in the five-to-fifteen-developer range, the math flips for most teams.

Compliance requiring ephemeral build environments. Some regulated industries — PCI, certain HIPAA configurations, FedRAMP — require build environments to be wiped between jobs so artifacts can't leak across runs. A persistent home agent fails that test. Microsoft-hosted's fresh-VM-per-job is what regulators want to see; alternatively a VMSS pool with delete-after-use achieves the same on self-hosted.

A genuine need for burst capacity. If your CI traffic is shaped like "twenty parallel jobs from 9 to 11am, idle the rest of the day," paying for twenty permanent self-hosted slots is silly when a KEDA-scaled container pool or a VMSS would handle it elastically.

Outside those, the case for self-hosting on hardware you already own is unusually clean. Capex zero or negligible. Opex single-digit dollars per month. Performance asymmetric upside — three to ten times faster in steady state — driven by mechanisms (warm caches, modern hardware, persistent Docker layers) that are durable rather than tied to any specific cloud provider's pricing decisions. The risk is bounded and operational rather than strategic.

For most solo developers and small teams, the question isn't whether to self-host. It's whether to keep paying for hosted while pretending the build time isn't a problem.