Batch-downloading artifacts from Atlassian Bamboo

Atlassian Bamboo is a CI/CD server intended for large organisations to run internal builds. One thing that the API designers forgot with this system was a way to enumerate and download artifacts from the CI server. Not easily anyway.

The Bamboo server has a reasonable REST API that does about 60% of the work for us, it’ll let us query what the last successful build was, and give us the metadata details of that, as JSON, which we can pick through to get out what we need: mostly this is just the build number.

With that, we can hit up the artifacts endpoint and download the rest, but there’s two flies in the ointment:

1. While the REST API supports Bearer authentication using a personal access token, the artifacts endpoint does not: one must use HTTP basic authentication
2. To download directory trees, one must scrape the HTML and walk the tree

Thankfully (2) can be achieved using wget --mirror.

# Bamboo server
: ${BAMBOO_URI:=https://bamboo.example.net}

# Output directory
: ${OUTPUT_DIR:=/tmp/output}

# Plan keys (with project key prefix)
: ${PLAN_KEYS:=PK-PROJ01 PK-PROJ02}

if [ -z "${BAMBOO_USER}" ]; then
	read -p "Bamboo username: " BAMBOO_USER
fi
if [ -z "${BAMBOO_PASSWORD}" ]; then
	read -s -p "Bamboo password: " BAMBOO_PASSWORD
fi

bamboo_wget() {
	wget 	--http-user "${BAMBOO_USER}" \
		--http-password "${BAMBOO_PASSWORD}" \
		"${@}"
}

# Download the latest artifacts from the following projects
for plankey in ${PLAN_KEYS}; do
	bamboo_wget	--header "Accept: application/json" \
			-O "${OUTPUT_DIR}/${plankey}-result.json" \
		${BAMBOO_URI}/rest/api/latest/result/${plankey}?buildstate=Successful\&os_authType=basic

	buildno=$( jq -r .results.result[0].buildNumber \
			"${OUTPUT_DIR}/${plankey}-result.json" )

	bamboo_wget -P "${OUTPUT_DIR}/${plankey}" --mirror -nd -np \
		${BAMBOO_URI}/artifact/${plankey}/shared/build-${buildno}/?os_authType=basic
done

# ${OUTPUT_DIR} will accumulate some cruft from `wget --mirror` but all the files
# should be there.

Hopefully it won’t be much longer and I won’t have to deal with this house of horrors much more, but for others that are stuck with it, here’s something that may help.

Tags: atlassianatlassian-bamboo