Hashing slow down computer too much

Archived discussion about features (predating the use of Bugzilla as a bug and feature tracker)

Moderator: Moderators

Locked
Cyborg
Posts: 41
Joined: 2004-01-23 18:15

Hashing slow down computer too much

Post by Cyborg » 2004-03-28 20:48

When i first installed and started 0.401 i didnt know what was wrong!
The CPU usage went up to 100% and stayed there!
I thought it might only last for a while 'til the program had been booting up and "settled down", but after an hour or so, it was still really slow.

It wasn't until i searched the forums for a while i found out that the program was hashing my shares (250GB).
I couldn't use my computer in any way while hashing! I couldn't even use winamp! it was reeeaally slow! (i've got an AMD XP 2600+ with 512 MB RAM).

You should display a popup windows or something similar every time the program starts hashing so the user knows what's going on (it's hard to notice the small statusbar at the bottom of the screen). And i also think you should lower the priority for it so one can use the computer in other ways while hashing.

PseudonympH
Forum Moderator
Posts: 366
Joined: 2004-03-06 02:46

Post by PseudonympH » 2004-03-29 01:13

The hashing thread is set to run at low (idle?) priority, so I don't know what your problem is... The only thing that it should affect is stuff that requires a lot of hard drive access. Anyway, once the hashing gets done the first time, you'll never have to wait for more than a few seconds at a time while new files are hashed.

vellu
Posts: 20
Joined: 2003-08-30 01:33
Contact:

Re: Hashing slow down computer too much

Post by vellu » 2004-03-29 02:01

Cyborg wrote:It wasn't until i searched the forums for a while i found out that the program was hashing my shares (250GB).
I couldn't use my computer in any way while hashing! I couldn't even use winamp! it was reeeaally slow! (i've got an AMD XP 2600+ with 512 MB RAM).
I've got a similar setup, with a smaller share though (about 50 gigs) and my system wasn't slow by any means during hashing. I didn't even notice it untill I tried opening another program, which opened a bit sluggish. I even had SETI running in the background which keeps CPU-load at full 100% all the time.

Maybe you had some other highly HDD intense task running at the same time?

Wisp
Posts: 218
Joined: 2003-04-01 10:58

Re: Hashing slow down computer too much

Post by Wisp » 2004-03-29 03:25

I didn't even notice that dc++ was hasing, later i found out there were hash ID's in my share..
I think a lot of people have configged their windows or hardware bad

TheParanoidOne
Forum Moderator
Posts: 1420
Joined: 2003-04-22 14:37

Re: Hashing slow down computer too much

Post by TheParanoidOne » 2004-03-29 03:34

Cyborg wrote:(it's hard to notice the small statusbar at the bottom of the screen).
How so? The names are constantly flashing across the status bar. It was the first thing I noticed.

I also have SETI@Home running on my desktop which is a PIII866. The only way I noticed that it was actually hashing was the flashing names in the status bar. It didn't affect me in any way whatsoever.
The world is coming to an end. Please log off.

DC++ Guide | Words

crazy_snake
Posts: 1
Joined: 2004-03-29 03:57

Post by crazy_snake » 2004-03-29 04:06

about hashing. i installed the new version of dc+ yesterday, and my old maxtor 120gig crashed with damaged clusters exactly where dc+ were hashing with difficulties...i think my hd crashed because it was old...but ..but.... groaarrr !!
is there a way that hashing damage hd (logical, there are more access but....) ?

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2004-03-29 04:23

That people would complain that hashing interferes with their computer use - well, expected, and I'm sure it does for many people. However, I hadn't expected someone to blame DC++ for faulty, failing hardware.

This is all quite amusing.

Guitarm
Forum Moderator
Posts: 385
Joined: 2004-01-18 15:38

Post by Guitarm » 2004-03-29 04:31

cologic wrote:I hadn't expected someone to blame DC++ for faulty, failing hardware.
Well, this is actually often the case when you have a forum for something. People always take the opportunity to blame something/someone for things that aren't actually related. I've seen this in support-departments for a long time. Something psychological :) . But I agree with you, it's kinda strange how people can blame DC++ for just about anything
"Nothing really happens fast. Everything happens at such a rate that by the time it happens, it all seems normal."

cyberal
Posts: 360
Joined: 2003-05-16 05:42

Re: Hashing slow down computer too much

Post by cyberal » 2004-03-29 04:53

TheParanoidOne wrote:
Cyborg wrote:(it's hard to notice the small statusbar at the bottom of the screen).
How so? The names are constantly flashing across the status bar. It was the first thing I noticed.
Well, not if you have long folder names (like me)... original releases tend to get long and since the only thing that differs between the files is the extension you can't see anything moving down there... maybe the path name could be cut down to fit? Probably hard to know how much space is available though?
http://whyrar.omfg.se - Guide to RAR and DC behaviour!
http://bodstrom.omfg.se - Bodströmsamhället, Länksamling om hoten mot vår personliga integritet

Cyborg
Posts: 41
Joined: 2004-01-23 18:15

Re: Hashing slow down computer too much

Post by Cyborg » 2004-03-29 08:25

TheParanoidOne wrote:
Cyborg wrote:(it's hard to notice the small statusbar at the bottom of the screen).
How so? The names are constantly flashing across the status bar. It was the first thing I noticed.
I didnt notice at all..it took dc++ like 1-2 minutes to hash each file.

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2004-03-29 08:37

Well, on my computer (Athlon XP 2100+/1.73GHz) DC++ hashes approximately 25-35MB/second depending on conditions. That's about 1.8GB/file to have to hash for a minute. Either your share consists entirely of enormous files, your disk system sucks (DMA mode enabled?), or your filesystem sucks (excessive fragmentation?).

Your computer sounds broken, and blaming DC++ is silly.

TheParanoidOne
Forum Moderator
Posts: 1420
Joined: 2003-04-22 14:37

Post by TheParanoidOne » 2004-03-29 09:09

crazy_snake wrote:is there a way that hashing damage hd (logical, there are more access but....) ?
Um, no. Hashing is not doing anything wild and wacky to your disk. It is reading the contents of a file, doing a calculation and then moving on to the next file.

DC++ would be reading the disk anyway, if people were uploading from you.

If your disk has bad clusters, that should be the warning sign for you to buy a new disk. You shouldn't being waiting until it actually stops working.
The world is coming to an end. Please log off.

DC++ Guide | Words

Cyborg
Posts: 41
Joined: 2004-01-23 18:15

Post by Cyborg » 2004-03-29 17:49

cologic wrote:Your computer sounds broken, and blaming DC++ is silly.
It's not broken! I just baught it and ALL other program/games works just fine! I've also used SiSoft Sandra to check the speeds of all my components and i compared them to average results by other users and they matched really good!

Todi
Forum Moderator
Posts: 699
Joined: 2003-03-04 12:16
Contact:

Post by Todi » 2004-03-30 01:36

Your answer is not really relevant. Try using another similar p2p program and see if it produces the same results, then you might have something to say. New harddisks are sometimes damaged when you get them, it's not that uncommon.

TheParanoidOne
Forum Moderator
Posts: 1420
Joined: 2003-04-22 14:37

Post by TheParanoidOne » 2004-03-30 04:38

I don't think it's a case of "broken" but more a case of misconfiguration such as not enabling and/or correctly configuring DMA.
The world is coming to an end. Please log off.

DC++ Guide | Words

azoapes
Posts: 1
Joined: 2004-03-30 13:35

Hashing... again....

Post by azoapes » 2004-03-30 14:09

I was thinking about, what's hashing REALLY abaout, like...what's improving whith this feature?
The only thing I can think of, is a shorter initializingperiod, but does that overweigh the fact that it takes about three hours to hash a 200GB share?

Just a thought...
I'm not stupid, I'm just having unlucky moments while I'm thinking... ~Pooh Bear, 1998

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Re: Hashing... again....

Post by GargoyleMT » 2004-03-30 21:36

azoapes wrote:I was thinking about, what's hashing REALLY abaout, like...what's improving whith this feature?
What's in it for you?

Congratulations, you've asked the most relevant question in the thread. (no, not sarcasm)

Hashing uniquely fingerprints the contents of a file - change a byte, and the calculated hash changes. But rename the file, and you can still know it's the same file. Ever collect videos, where everyone has their own naming scheme? Finding alternate sources and adding them manually should be a very rare occurence when everyone has hashed their shares.

Plus, hashing will enable checking to see if your downloads are being corrupted in transfer (and fix them if they are).

It will allow you to search for something, go on the web, and see if other people think it's a complete file, or a fake one.


There. Those are the major points. If you search around the forum, you may find other nuggets as well.

Cyborg
Posts: 41
Joined: 2004-01-23 18:15

Post by Cyborg » 2004-03-30 22:20

TheParanoidOne wrote:I don't think it's a case of "broken" but more a case of misconfiguration such as not enabling and/or correctly configuring DMA.
Might be. DMA is enabled on my computer, but i dont know how to configure it.

GargoyleMT wrote:Hashing uniquely fingerprints the contents of a file - change a byte, and the calculated hash changes.
Not really. In that case, you've created the most powerful compression one could ever think of. If what you claim is true, that would mean that one could reverse the hashing to get a huge file of just a small set of characters.

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Post by GargoyleMT » 2004-03-30 22:36

Cyborg wrote:Not really. In that case, you've created the most powerful compression one could ever think of. If what you claim is true, that would mean that one could reverse the hashing to get a huge file of just a small set of characters.
http://www.cs.technion.ac.il/~biham/Rep ... tiger.html
http://www.geocities.com/fra_u_d/ (caution, this is humor)
http://www.md5crk.com/?sec=whatismd5crk
http://www.iks-jena.de/mitarb/lutz/secu ... q/q99.html

Please read, or at least skim.

Hashes do not imply reversibility.
What you're referring to (two files having the same hash) is called a collision.

dpm
Posts: 10
Joined: 2004-03-30 22:40

Post by dpm » 2004-03-30 23:22

TheParanoidOne wrote: Um, no. Hashing is not doing anything wild and wacky to your disk. It is reading the contents of a file, doing a calculation and then moving on to the next file.
I'm sure this is fine for all of you users who share less than 19982 files with a total size of 271 GB

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2004-03-30 23:26

Elitism really works better when you're elite.

20k files of 271GB total size is not close to special or particularly unusual in DC; I've had twice that, for example, and there were always much larger shares than mine around.

Despite that, hashing has worked quite well for me.

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Post by GargoyleMT » 2004-03-30 23:58

cologic wrote:Elitism really works better when you're elite.
:)
dpm wrote:I'm sure this is fine for all of you users who share less than 19982 files with a total size of 271 GB
Works great for me: 39,081 438,95GB.

I'm not a huge sharer, but I am a dev. Hashing my share takes overnight, but even when it's not timed well, I can use my computer just fine.

dpm
Posts: 10
Joined: 2004-03-30 22:40

Post by dpm » 2004-03-31 00:29

GargoyleMT wrote: Works great for me: 39,081 438,95GB.

I'm not a huge sharer, but I am a dev. Hashing my share takes overnight, but even when it's not timed well, I can use my computer just fine.
So I take it you too are running about 80 or so processes normally?
(this is far from the problem. I run two direct connect hubs, apache/php, mysql, all of the normal programs one uses, etc. Too many processes is not a problem when there is no hashing)

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2004-03-31 00:40

I run a DC hub, Apache/mod_python, MySQL, all the normal programs one uses, etc.

And, oh, wait, it works perfectly.

dpm
Posts: 10
Joined: 2004-03-30 22:40

Post by dpm » 2004-03-31 01:01

cologic wrote:I run a DC hub, Apache/mod_python, MySQL, all the normal programs one uses, etc.

And, oh, wait, it works perfectly.
I don't call 8 GB an hour consuming 30% or, 1 GHz of processing power "fine"

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2004-03-31 01:17

8GB/hour of tiger hashing consuming 30% of (presumably a 3+GHz Pentium 4?) strikes me as implausible. Here's why:

This site provides some hash speeds that have proven fairly reasonable in my experience. With that in mind, the 8GB/hour number is equivalent to 2.27MB/s. Compare this with 40MB/s for Crypto++ on a (again, presumably) lesser machine than yours.

There's an order of magnitude difference. Can you explain this?

dpm
Posts: 10
Joined: 2004-03-30 22:40

Post by dpm » 2004-03-31 01:35

So far, since 9pm, it appears to have hashed all of 21 GB.

This is assuming it hashes alphabetically through my share, recursively, which would lead it to start at Download/250GB/Anime/Chobits/

And ack, there seems to be no way to upload a file.

Image

ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Post by ivulfusbar » 2004-03-31 01:48

CPU usage of a dc++ client is dependent on three important factors. The number of users you faces, the number of searches you recieve and the number of files you share. Ofcourse Number of users and number of searches can be considered highly correlated.. Its not the amount in size that is important.. its the number of files.. So your picture are utterly useless.
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

OrangeSlice
Posts: 12
Joined: 2003-12-09 19:53

Post by OrangeSlice » 2004-03-31 01:49

dpm, hashing might be going very slow for you because you have all those other processes taking up cpu time and since the hashing thread priority for dc++ is set to low all other processes running on the system get priority over it.

see what happens if you close everything else then run dc++

dpm
Posts: 10
Joined: 2004-03-30 22:40

Post by dpm » 2004-03-31 02:05

OrangeSlice wrote:dpm, hashing might be going very slow for you because you have all those other processes taking up cpu time and since the hashing thread priority for dc++ is set to low all other processes running on the system get priority over it.

see what happens if you close everything else then run dc++
It's not the slowness that matters to me. I question the fact that it is running on low priority. Setting the DCPlusPlus Process itself to a lower priority eliminated the blips I had, along with restricting it to running on just one "processor" (I have HyperThreading) Closing other programs is hardly an option, as 600 other people depend on my computer being on 24/7 (as you can tell by the two Direct Connect Hubs running) They also take very little processor power.

vellu
Posts: 20
Joined: 2003-08-30 01:33
Contact:

Post by vellu » 2004-03-31 02:10

Ziisas, DPM.

You have a whopping 80 processes running and you're complaining about high CPU usage? I don't think I could even find 80 tasks to run on my system... (unless I'd just randomly start up programs unnecessarily)

And since DC++ hashing is Low Priority, no surprise it's going to take some time.

OrangeSlice
Posts: 12
Joined: 2003-12-09 19:53

Post by OrangeSlice » 2004-03-31 02:10

acquire dedicated server, problem solved

vellu
Posts: 20
Joined: 2003-08-30 01:33
Contact:

Post by vellu » 2004-03-31 02:13

vellu wrote:Ziisas, DPM.

You have a whopping 80 processes running and you're complaining about high CPU usage? I don't think I could even find 80 tasks to run on my system... (unless I'd just randomly start up programs unnecessarily)

And since DC++ hashing is Low Priority, no surprise it's going to take some time.
Strike that....sleepy eyes should've read the thread more carefully....

dpm
Posts: 10
Joined: 2004-03-30 22:40

Post by dpm » 2004-03-31 02:19

vellu wrote:Ziisas, DPM.

You have a whopping 80 processes running and you're complaining about high CPU usage? I don't think I could even find 80 tasks to run on my system... (unless I'd just randomly start up programs unnecessarily)
It's easier than you think.

There's about 20-30 default Windows processes. DC processes number another 3. Of course there is iTunes, which comes with the iPodHelper service, mIrc, Trillian, TorrentStorm (bitorrent) Apache, mySQL, explorer, iExplore, CoolMon, rainlendar, daemon tools, UltraMon, I mean, I don't see how you can have less than 60 processes..

ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Post by ivulfusbar » 2004-03-31 02:23

You still haven't answered my post in this thread.. The cpu-usage is not interesting until you have answered it.
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

vellu
Posts: 20
Joined: 2003-08-30 01:33
Contact:

Post by vellu » 2004-03-31 04:10

dpm wrote:It's easier than you think.

There's about 20-30 default Windows processes. DC processes number another 3. Of course there is iTunes, which comes with the iPodHelper service, mIrc, Trillian, TorrentStorm (bitorrent) Apache, mySQL, explorer, iExplore, CoolMon, rainlendar, daemon tools, UltraMon, I mean, I don't see how you can have less than 60 processes..
True enough...didn't really read through the thread carefully enough before posting earlier...duh.

On a typical workstation though 80 is a lot. My system (win xp pro) runs after reboot with firewalls/antivirs/few miscellaneous at 40 processes. To get that up to 80 would require quite the multitasking madness without starting any extra servers/services.

Typically my system stays at about 45-48 processes when doing what ever it is I happen to be doing. Hardly ever over 50.

Yorthen
Posts: 1
Joined: 2004-04-05 16:09

Post by Yorthen » 2004-04-05 16:19

vellu wrote:
dpm wrote:It's easier than you think.

There's about 20-30 default Windows processes. DC processes number another 3 ....... I don't see how you can have less than 60 processes..
On a typical workstation though 80 is a lot. My system (win xp pro) runs after reboot with firewalls/antivirs/few miscellaneous at 40 processes. To get that up to 80 would require quite the multitasking madness without starting any extra servers/services.

Typically my system stays at about 45-48 processes when doing what ever it is I happen to be doing. Hardly ever over 50.
It sounds like you both should spend some time going through which services you are running, after that take a look at the other programs you are running, are they all necessary? When my Win 2003 boots up I'm only running 24 processes. Running a lot of programs and/or servers/daemons I might reach 50 but then I wouldn't go complaining if the computer was a bit slow. Expecting anything else is unrealistic.

--
Edit
The bad part about finding post via the search-function is that you might contribute to a dead conversation :(

vellu
Posts: 20
Joined: 2003-08-30 01:33
Contact:

Post by vellu » 2004-04-06 03:17

Yorthen wrote:It sounds like you both should spend some time going through which services you are running, after that take a look at the other programs you are running, are they all necessary? When my Win 2003 boots up I'm only running 24 processes. Running a lot of programs and/or servers/daemons I might reach 50 but then I wouldn't go complaining if the computer was a bit slow. Expecting anything else is unrealistic.
Oh no, I'm (we're) not complaining...My system runs perfectly with the processes I have.

After a closer look I find the following (currently running 45 procs):

There seems to be 22-23 processes that always start. On top of that there are some driver related procs (graphic card, printer, scanner, cdrw packet burning INCD, mouse) which add 8 procs, which gives 30 procs that cannot be avoided. Then it is absolutely necessary to start antivirus and firewall programs, which actually add quite a few processes, 8 total.

This would give 38 procs always. After that it's non essentials like DUMeter, Daemon Tools, Seti, DC++, TVS, IExplorer (+taskmgr) for a total of 45 at the moment.

And with these running (including Seti which always causes 100% CPU utilization) I hardly noticed the Hashing going on.

Locked