Google Answers Logo
View Question
 
Q: Push/pull ideas for moving big files on an Extranet? ( No Answer,   4 Comments )
Question  
Subject: Push/pull ideas for moving big files on an Extranet?
Category: Computers > Internet
Asked by: humptydank-ga
List Price: $50.00
Posted: 30 Jul 2004 20:23 PDT
Expires: 29 Aug 2004 20:23 PDT
Question ID: 381578
Hello!

We are a small company with limited resources, but have offices in six
countries with a main office in New York.  We receive a tremendous
amount of materials from our clients on cd in New York, and we need to
distribute them to our outer offices in some easy fashion.  We've run
into the following barriers:

1)  Making them all available for download all the time would get us
into godzillabytes of storage in a month, so we'd just like to make
them available for download for a little while, let them cut cds
locally, and then take them down.

2)  We don't want to count on someone in the outer office to remember to
initiate a download overnight in order to get the files.

3)  Since these files are being downloaded (over broadband, yes) from
New York to Europe, the potential for interrupted downloads is fairly
high.

We have envisioned some sort of push or synchronization system whereby
we post the files on a local server, and they are automatically
distributed to machines on the outer offices, or the outer office
machines periodically poll the New York server for changes and get any
new files.  Each of these actions would need to happen automatically
at a specified time.

Some factors:

-- The individual files will be of varying sizes, but the total amount
of information will be large.  I'm sitting, for example, with an 11 cd
set in front of me that needs to go out.  So it would need to be able
to move large amounts of data reliably and unattended.

-- I don't have to install or run any files after they are moved, just
move them like ftp.

-- We'd like the transfers to happen at night, but we don't want to
have to remember to initiate transfers or have them remember to
download, so sending should be automated.

-- Automatic resume would be a must.

-- All the machines involved are Windows.

-- Low cost:  We're trying to save money with this system over
shipping physical cds, so I'm looking for a cool hack, it doesn't have
to be super-robust enterprise stuff.

-- Low geek-factor remotely:  We could have the outer offices install
ftp servers and such, but that's over their heads and tough for me to
walk them through it, and it raises security issues etc.  I was hoping
for something that had a simple install on the remote end and left any
heavy geek stuff to me.

Sooo, like I said I'm looking for cheap/free and/or cool hacks to get
this done, and I'm geeky enough that I could probably script a
solution if someone points me in the right direction.  I'm hoping that
there might be tools built into Windows that I could somehow bring
together to get this done.

Thanks!

-- Dave

Request for Question Clarification by sublime1-ga on 30 Jul 2004 21:49 PDT
humptydank...

How about a download controller which can be installed at
all your outer offices, and set to download at the same
time every day? Filenames for the day could be emailed
before close of day, and the downloads set to go during
the night.

For example, if the ftp address was ftp://yourcompanyIP/
you could send the filenames of this.exe and that.zip,
and they could enter the addresses:
ftp://yourcompanyIP/this.exe
ftp://yourcompanyIP/that.zip

...then set the program for autodownload and set the 
time for 11pm or so. Download Mage also allows multiple
threads (user defined) for the same file(s), which, on
my system, is set to 4, and effectively quadruples the
speed of the site's bandwidth. It will also autoresume
if interrupted. It's pretty intuitive to configure.

Download Mage:
http://www.dlmage.com/

sublime1-ga

Request for Question Clarification by sublime1-ga on 06 Aug 2004 20:30 PDT
So Dave...

Have we gotten you anywhere on this?

sublime1-ga

Request for Question Clarification by pafalafa-ga on 06 Aug 2004 20:42 PDT
Dave,

Maybe I'm oversimplifying things, but it seems to me you could just
email the files using this service here:

https://s2.yousendit.com/default.uplx

Yousendit is a free service, and will store as much as 1 gig per email
for seven days--your remote offices are notified by email as new files
are posted for download.

Not fully automated, but it seems pretty foolproof just the same.


Like I said, maybe Im oversimplifying...then again, maybe not.  Have a
look and let me know.

pafalafa-ga
Answer  
There is no answer at this time.

Comments  
Subject: Re: Push/pull ideas for moving big files on an Extranet?
From: mik737-ga on 30 Jul 2004 21:04 PDT
 
Looks like your going to get a free answer.
I suggest you use a simple VB program. You are looking for some way to
send or get the data accross the internet/extranet automatically. This
will take some programming to do it either way. A simple VB (Visual
Basic) program can be made in about an hour by a professional. This
program would simply run on each of the remote computers. When a
specific time is set, the program will download from your host
computer ftp site a text file with file names and file sizes. Once the
program has determined what to download it would start to download
each file large file. After the files have been completely downloaded
it, the remote computer would send a message to the host that it has
successfully completed the download.
Subject: Re: Push/pull ideas for moving big files on an Extranet?
From: crythias-ga on 30 Jul 2004 23:34 PDT
 
:) don't forget DVDs which hold 4.7GB if you still want to ship your
info. Same form factor as CDs, but you get up to 7x more storage per
disk. So, you'd ship two DVDs instead of 11 CDs. Shipping costs go
down drastically. And you only need a $35 DVD reader.

Or, backup/restore from tapes, ... one tape can hold 40GB (or more).

But, yeah, the other solutions would be the way to go.
Subject: Re: Push/pull ideas for moving big files on an Extranet?
From: forge-ga on 03 Aug 2004 11:03 PDT
 
FEST, a little tricky to get setup, but rock solid once it's going.
Have had it running for 2 years in an enterprise environment sending
very large business critical files and it has only had 2 issues.

http://www.softlink.com/fest_techie.html
http://www.tkg-usa.com/fest.html

Can't comment on price, but it is a great piece of software.

forge
Subject: Re: Push/pull ideas for moving big files on an Extranet?
From: buzzau-ga on 08 Aug 2004 22:42 PDT
 
Unison is a Free OpenSource project, we use this and it is very successful

http://sourceforge.net/projects/unison/

Unison is a file-synchronization tool for Unix and Windows. (It also
works on OSX to some extent, but it does not yet deal with 'resource
forks' correctly; more information on OSX usage can be found on the
unison-users mailing list archives.) It allows two replicas of a
collection of files and directories to be stored on different hosts
(or different disks on the same host), modified separately, and then
brought up to date by propagating the changes in each replica to the
other.

Regards
Simon

Important Disclaimer: Answers and comments provided on Google Answers are general information, and are not intended to substitute for informed professional medical, psychiatric, psychological, tax, legal, investment, accounting, or other professional advice. Google does not endorse, and expressly disclaims liability for any product, manufacturer, distributor, service or service provider mentioned or any opinion expressed in answers or comments. Please read carefully the Google Answers Terms of Service.

If you feel that you have found inappropriate content, please let us know by emailing us at answers-support@google.com with the question ID listed above. Thank you.
Search Google Answers for
Google Answers  


Google Home - Answers FAQ - Terms of Service - Privacy Policy