TheGeekery

The Usual Tech Ramblings

PowerShell and BITS

Ever had to download a bunch of files from a website, and didn’t want to have to write an HTTP handler for it? This comes up on a regular basis here. We get requests from customers that have transferred from one vendor to another, and want to import all their photos to us. Here is how I solve it.

What is BITS?

BITS, or Background Intelligent Transfer Service, is a service bundled with Windows that is used to transfer files from websites. It was originally introduced in Windows XP RTM back in October 2001. You may not know it, but your computer uses it all the time if you have Windows Automatic Updates enabled. It is an intelligent service that asynchronously transfers files, automatically adjusting bandwidth usage, throttling downloads, as bandwidth becomes available. It can even be used to download files across computer starts.

How is it useful in PowerShell?

BITS is really easy to use, a few commands which I’ll show shortly, and you can download a bunch of files quickly, without having to write HTTP handler. In my case above, I receive text files, with a list of image URLs, that need to be imported.

Here is how to do a very simple, single file, transfer.

Import-Module BitsTransfer
$job = Start-BitsTransfer -Source http://somedomain.com/somefile.jpg -Destination c:\images\somefile.jpg -Asynchronous
while( ($job.JobState.ToString() -eq 'Transferring') -or ($job.JobState.ToString() -eq 'Connecting') )
{
	Sleep 3
}
Complete-BitsTransfer -BitsJob $job

Relatively simple, a few lines of code for a single file. However, BITS really shines when you’re working with multiple files to download.

As the files I receive are quite often comma-separated values (CSV) files, we can use the rather handy Import-CSV function, and we build up from there.

Import-Module BitsTransfer
$filedata = Import-CSV c:\images\import.csv
$filedata | ForEach-Object {
	$vin = $_.VIN
	$images = $_.Images
	
	$job = $null
	
	"Processing VIN: {0}" -f $vin
	$img_split = $images.Split('|')
	for($i = 0; $i -lt $img_split.Count; $i++) {
		if ($img_split[$i].Length -eq 0) {
			continue
		}
		
		$outpath = 'C:\Images\{0}_{1}.jpg' -f $vin, $i
		
		if ($job -eq $null) {
			$job = Start-BitsTransfer -Source $img_split[$i] -Destination $outpath -Asynchronous
		} else {
			Add-BitsFile -BitsJob $job -Source $img_split[$i] -Destination $outpath
		}
	}
	if ($job -ne $null) {    
		while( ($job.JobState.ToString() -eq 'Transferring') -or ($job.JobState.ToString() -eq 'Connecting') )     
		{
			Sleep 3     
		}     
		Complete-BitsTransfer -BitsJob $job
	}
}

This is a little more complicated, but is still pretty simple. It introduces Add-BitsFile which allows you to add files to an existing job, and a simple ForEach loop to go through an array of URLs.

I’ve made a few assumptions here, and thrown out all kinds of error handling in favor of a quick code turn around. For example, the job state of a BITS job can be one of nine options (which you can see here), but I assume only two. I’m also not handling cases where I exceed the number of jobs, but that never happens because I only ever have one job running at a time. I’ll probably work on tidying it up a little for other people’s use, but for now, it’s just me using it where I work, so the impact of an error is very minimal.

Give it a shot, download some images off of Flickr1, or go a little further, and test it against downloads from Microsoft.

Got any questions? Hints? Tips? Leave them in the comments, I’d love to hear from you.

  1. If they are yours, or have Creative Commons license 

Comments