What's new
TerraForums Venus Flytrap, Nepenthes, Drosera and more talk

Register a free account today to become a member! Once signed in, you'll be able to participate on this site by adding your own topics and posts, as well as connect with other members through your own private inbox!

Is faster file copying possible?

  • Thread starter swords
  • Start date
Hey computer folks, I've been working at installing my huge synth bundle (around 300GB between 7 synths) and it's taking forever! Each program's sound library is about 30 GB and the installer usually craps out so most people drag and drop a few folders a day into the sound libraries (folders) once the program and file allocations on disk #1 install but it's so slow even this way! I have to dedicate at least a whole night to installing one program, so if I'm lucky and Buddha is smiling upon me, I can get one set installed in one night. Symphonic Choirs worked that way anyway. The Brass instrument disk for my orchestra is messed up so they're sending me a new one that should be here by noon...

Anyway, my question is: Is there any way to make the computer put all its resources into the file transfer I have a quad core and basically nothing runs on this PC (just enough processes to keep it conscious) so shouldn't it be moving the files faster?


As an aside, are the external Hard drives any good? Are they like Sata2 speed (7200 RPM)? Any recommendations on brand? I'd sure like to move all the huge sound libraries to a dedicated USB HD and NEVER have to sit through this again and only have to setup the actual program installation which only takes like 8 seconds. :(
 
The quad core (your processor) is doing virtually no work on making the installation happen. Copying files doesn't take computations - just disk access. Having a particularly fast processor doesn't do much of anything for disk-based input/output, since processors have pretty much always been orders of magnitude faster than drives (both hard drives and CD/DVD drives.) Since your RAM is probably already pretty big, the best way to speed things up is to get faster drives and/or drive controllers. It could also be that you're bumping up against your motherboard's bus speed, but I think that's unlikely.
Disable or kill any processes that might be using your hard drive at the same time as the installer. This is usually hard to do, since your operating system is often on there, as well as your swap space. If Vista has an option for turning off virtual memory (hehe, seems unlikely somehow...) you could get some more disk time out of that. Also, it might be worth noting that unless you're hand-copying the files directly to the place they're supposed to go on the hard drive, it will be faster to just install from the application disk (theoretically.)
On the bright side, you can do pretty much any lightweight task that doesn't use the drives intensively without worrying about slowing the installer down significantly. Web browsers, word processors, etc. should all be fine. Avoid stuff that hogs the processor a lot, especially stuff that manages to use more than one core at a time. The installer spends most of its time waiting for the disks, but the processor can still become a bottleneck if you bog it down with something else.
~Joe
 
Theoretically is the operative word here!

CPU usage is generally at 0% then jumps to 15 then 5 then 0. WHen the installer is running it goes up to 20-25% and files fly by but as I say, it seems to crap out on just about everyone. If you check out the soundsonline forum (forum for this software) most people end up manually installing the libraries the way I am cos the installer eventually just quits. I can't imagine the people who sit through the installation of the 33 disk platinum orchestra! 6 was bad enough! The company now sells "diamond editon" of their hollywood strings library already on a HD you just install the program itself and point the library directory to the external drive. They only charge like $1000 extra for that though... :censor:

My HDs are 7200 RPM sata and the DVD is 48X I believe.

Can I get faster drives that are external and USB connectible? Like a really fast dual layer DVD reader and a really fast external drive? Is that possible? Any advice on what I should look for in faster drives? I'd pay it just to be able to get things moving along and start playing (esp. considering what I paid for the damn software bundle). The software runs good once you figure out what goes where with the "virtual midi cables" that link one program to another but it's just getting the enormous libraries onto the drive.
 
USB is a speed bottleneck. You want an eSATA II or Firewire drive if you're going to use an external drive, provided your motherboard supports these (connectors). If not you have to buy a card.

Having drives attached as primary and secondary can slow transfers down. Attach them all as primary if possible.

USB 2.0 has a transfer rate of 480 Mbits/s (60 MBytes/s) vs eSATA II 3 Gbits/s (384 MBytes/s)

Firewire transfer rate 400 Mbits/s - however Firewire uses a peer-to-peer architecture vs USBs master/slave architecture which in reality leads to faster transfers.
http://www.cwol.com/firewire/firewire-vs-usb.htm

Of course there is SATA 3 (6 Gbits/s), Firewire 800, Firewire 3200, and USB 3.0 (5 Gbits/s). There is also Serial Attached SCSI (SAS). You'd have to buy a card for any of these (some are not available yet). SAS drives cost 1.5x-3x as much as SATA drives and the controller card will probably set you back for at least $200.

File operations do consume CPU usage. Any disk benchmark program worth its weight in salt will report CPU usage. USB and Firewire consume the most - traditionally SCSI uses the least (that's one of the reasons why it is so expensive, lower error rates is another).

The New Technology File System (NTFS) that Microsoft uses has always been somewhat slow. IBMs High Performance File System (HPFS) used in OS/2 was faster. Unix file systems are much faster yet. And as I recall over complaints of the number of times dirty shut-downs occurred with Windows 2000 (resulting in disc scans at startup) Microsoft erred to much too side of caution and slowed down the file writes and shutdown in XP. This was tweaked in service packs.

This might help:
Improve NTFS performance

You could also try setting the processor affinity, though I doubt it would help much.
 
Thanks Nan, I've seen the eSataII externals, but I think even these are only 5200RPM I do have one firewire port and have never used it for anything.

When you say attached as primary you mean installed internally correct? I was really hoping to avoid that! lol

What would be neat is a seperate PC case full of smaller drives and a few big honking fans that I could hide under my desk and somehow run up to the case with the motherboard and all.

In regards to a faster Dual Layer DVD Rom reader is USB OK for use here? I don't know how fast mine reads DVD roms (It's about 3-4 years old now) but most of the available new ones seem to read DVD roms at 8x.
 
File operations do consume CPU usage. Any disk benchmark program worth its weight in salt will report CPU usage.

Well, yeah, but mostly in the sense that almost everything the computer does takes some CPU time. The install program has to be interpreted, but most of its actual "work" in terms of real execution time is happening on the disks - it's not like the process spends time waiting on the CPU. (Compare to... I dunno... BLAST? Compilers? Photoshop when the files don't exceed the size of active memory?) My point was that it was highly unlikely that the processor was the bottleneck.

When you say attached as primary you mean installed internally correct? I was really hoping to avoid that! lol

I'm not sure I've ever come across those terms. Perhaps it refers to master/slave settings? Haha - looks like I guessed right.
As for the USB DVD drive, check to make sure that it's USB 2.0. It will probably be slower than your internal optical drive anyways, but if it isn't USB 2.0 then it will likely be unacceptably slower. But you might try it just to find out - it sounds like this process is slow enough that a delay for a little experiment would be a drop in the bucket.
~Joe
 
Really? You think a new DVD drive could actually be slower than the one already onboard even if the drive claims to read at 8x for DVD Roms? I definitely don't wanna pay more to do this any slower! lol

Got my replacement Brass collection but once again it crapped out at 50% right on the crescendo sample in the solo trumpet folder same as the earlier disk... :censor:

I am currently able to drag/drop the files again however, so that's where I'm at.
 
I thought you said it was three or four years old. I'm mostly suspicious of the connection you'll be using for it. If it were Firewire 800, or eSATA, I'd have more confidence, but USB isn't really ideal for high-performance uses.
This whole thing is on dual-layer DVDs? Can you get them on smaller ones? Conventional DVD drives are cheaper and typically faster than dual-layer drives.

What would be neat is a seperate PC case full of smaller drives and a few big honking fans that I could hide under my desk and somehow run up to the case with the motherboard and all.

They make those - probably won't solve your problem though.
~Joe
 
You may try raising the priority of the process to "High" in Task Manager. This may provide marginal gains of your installer if there are many processes and services running on your box. Just be careful not to set it to "Realtime".

This probably won't help though if the installer is just a shell and is spawing other processes during the install as is common. This is a little out of my area as a DBA.
 
  • #10
Well this is silly... I've noticed that wiggling my mouse periodically makes the files transfer faster and not freeze up. OK! Well that's what I'll do then. It's certainly cheaper than buying new drives! I always though people wiggling the mouse was just a nervous habit like my lip biting.

As I say I do have a firewire slot. I had it built in for an external digital recorder/mixer I was looking at but never purchased. Can say 5 or six external HDD be linked together and run into the single firewire lot? I see they sell External drives the shape of a hardcover and can be stacked side by side, I like that! I could situate them away from the PC by the widow where it's much cooler.

How difficult is it to install a SATA slot? I have a number of unused PCI slots on my case.

Almost everything I have runs on USB 2.0, QWERTY keyboard, MIDI synthesizer, Wacom pen tablet, etc. But it's just no good for drives? Either optical or harddisk? I was looking at the eSATAII at Best Buy today and it appears I could hook it up to my PC through the firewire. I see even seagate has 7200RPM external drives But they are USB 2.0. I'm hoping to get some answers from the eastwest forums about external drives but nobody is responding by even saying I'm being foolish to even ask about it. Like they're keeping it a secret or something! lol
 
  • #11
You'd probably want to add an eSATA card for one of those external drive enclosures. Firewire is a little on the slow side and you could do better. The good enclosures come with special PCI controller cards built specifically for managing that connection. Installation isn't hard at all - usually a three- or four-step process, not much more difficult than adding a hard drive and only slightly more involved than popping a new RAM module in.
Look for 10,000 RPM drives. They're more expensive, but probably what you want. Also, two other measurements you'd be interested in are seek time (you want a low number here) and data throughput (I can't remember the usual name for it - it's the top speed that data can be read in or out - you want a high number for this one, obviously.)
For the enclosure, if it doesn't break your budget, try to get one that does RAID. With the right flavor of RAID configuration you can get better performance.
As for wiggling the mouse, it could be that some background process that starts when the computer is "idle" is interfering with the installer. Do you use a screensaver? I suppose Vista might be running some drive indexing or other tasks in the background when it thinks there's nothing else to do.
~Joe
 
  • #12
I'm on XP 32 bit, not Vista. I have the screen saver off but the PC always shuts itself down after like 30 minutes of inactivity(even though files are moving), some "power saving" feature I suppose. when I was using XP 64 it didn't do that. I may revert back to x 64 and add 4 GB more of ram but first I wanna get these libraries onto separate drives.

These dual layer DVDs don't read at all on the Best Buy Vista laptop, is that some error in vista or is it just that it's the cheapest DVD drive they could stick in the thing?
 
  • #13
Vista, XP... I just can't keep track. Either way, it's likely that Windows :D is trying to find something better to do than twiddling its thumbs while it thinks you aren't using the computer.
It may be that your laptop doesn't have a dual-layer drive. I think my laptop does dual-layer disks, but my desktop that's five or six years old certainly doesn't.
As for the power-saving feature, you should be able to turn that off. Without a Windows box in front of me, it's hard for me to say how, though.
~Joe
 
  • #14
I just checked the drives are set to never shut down during inactivity, just shut off the monitor after 20 mins, I turned that to "never" as well even though turning off the monitor shouldn't really interfere.

I've finally got the entire Brass samples installed, I can either have a coffee and relax or tackle one of the other 4 programs... Installing this software is becoming a hobby in itself!

---------- Post added at 08:15 PM ---------- Previous post was at 07:38 PM ----------

I had to activate the software after installation so since I already had the dongle in the USB slot (can't have it in while you are installing the programs for some reason) I decided to see what this fancy orchestra set sounds like... HOLY ****! The Wagner Trumpets make you pee yourself with fatherland pride! The enormous 70 piece string ensembles are velvety smooth and grand, solo violins are weepy and warbly just like the real deal (use the mod wheel for even more realism) I'm gonna have some fun with this and I've only sampled a few sounds, there are hundreds more instrument articulations to check out.

Yamaha, Roland, Korg and those others from my past may as well just give up making synths and sound samples! lol
 
Back
Top