As I am sitting here reviewing a project I have to do, I am trying to figure out why I have to write a project that is fairly complex because the tools available are limited due to the operating system.
The project goal is to synchronize two servers, or more specifically two directories containing over 100GB worth of images. The issue is, if they were next to each other, it’d probably be pretty easy with RoboCopy which is part of the Windows 2000 Support Toolkit. The problem is, one server is in Atlanta, GA, the other is in Mesa, AZ. Slightly far apart, in fact roughly 1,800 miles. The only option available is FTP.
So, I have to find some kind of way of synchronizing 100GB over FTP. So I start looking about. There appears to be a whole bunch of programs that can sync over FTP which is good, however, none of them appear to do it well. They appear to search, and store all the search results in memory. This is fine, for small amounts of data, but not 100GB of jpeg files. The best one I found managed to do “okay” until it got to about 1/4 of the way through, and it had eaten 150MB of memory. So I’m having to write my own. Having had several years of Delphi experience, and playing with some pretty cool components in it, I know exactly how I want to do it.
To the rescue comes the TClientDataSet component. This component is a very good at handling data. What’s even better is it can save to XML. That’s great because it means I don’t have to keep reprocessing information that hasn’t changed. So I start working on this sync program, and I think to myself…
This would be so much easier in Linux
Why? Because people have already written tools for synchronizing whole trees of data. Take rsync for example. The whole goal of that is to… yes… you’ve guessed it… synchronize data.
And why won’t they let me use Linux? Because they use Microsoft, and that is all they want to use. This would be a perfect example of how to put open source, and Linux to a good project… but hey… I’m not bitter or anything. I’ve just got a blinding headache from god knows what, and looking at code in a pointless project.