Hash managing not acceptable

Use this forum to flesh out your feature request before you enter it in <a href="http://dcpp.net/bugzilla/">Bugzilla</a>.

Moderator: Moderators

Locked
mrtoad
Posts: 3
Joined: 2004-10-25 08:30

Hash managing not acceptable

Post by mrtoad » 2006-02-16 12:43

:evil: Sorry, but it isnt. Something has to be done about hashing in DC++.

I am sure no one that store their shit localy even bothers, but for those of us who keep our shit on fileservers hash indexing seriously damages dc++ stability.

Hash Indexing by itself is a good idea. However, given all immensly poor fileserver protocols we have to use hash is not always that pleasant. Any idea how long tiem ita takes to hash 0.1TB from samba? Well, even with a samba server running at top speed it takes around eight (8!!!) hours. Eight friggin hours! Given samba is slower like crap it usually takes around 10-12 hours.

This is really an issue for samba to fix, but they seem very content with samba performing at this horrid speed so I dont expect anything to happend there.

Unless you are aware of it, I am pretty sure you are, but every time Windows screws upp and fails to connect a network unit DC++ will start with 20-40% of your actual share and start rehashing (removing in this case) your whole hash index. And I doubt any one is surprised by windows crapping out on its regular bootup tasks like for example connecting network units.

Id REALLY APPRECIATE it if the hashfiles was locked down after hashing because spending 10 hours rehashing your files every now and then is just painful. Then you would manually update them whenever there was a need for it - that way you would put the controls back into the hands of the users (thats where they belong anyway) and these unfortunate mishap wouldnt need to happend ever again.

(Yes, DC++ just started rehashing my gigazilion of data because my fileserver was disconencted by XP - I am not counting on being able to use my computer for antyhing else useful before next weekend or something like that :P) And no, I wont stick to Kazaa or bittorent. This is a flaw that needs attention, thats why I am writing this post... you already know well enough dc++ kick ass in many other areas.

ullner
Forum Moderator
Posts: 333
Joined: 2004-09-10 11:00
Contact:

Re: Hash managing not acceptable

Post by ullner » 2006-02-16 13:45

mrtoad wrote:This is really an issue for samba to fix, but they seem very content with samba performing at this horrid speed so I dont expect anything to happend there.
Right... So we should fix other's poor code? Or what? Why are you asking for an improvement of DC++ when the flaw is in Samba?
mrtoad wrote:every time Windows screws upp and fails to connect a network unit DC++ will start with 20-40% of your actual share and start rehashing (removing in this case) your whole hash index.
If the drive letter changes, yes, of course files will be hashed again. So, does the drive letter change?
mrtoad wrote: [...] Then you would manually update them [...]
Why don't you just set 'Auto refresh time' to 0? Then DC++ won't refresh the share when you've started DC++. (It will during startup.)

PseudonympH
Forum Moderator
Posts: 366
Joined: 2004-03-06 02:46

Post by PseudonympH » 2006-02-16 13:47

Unless explicitly told to rebuild, DC++ will not remove entries from the hash database. When the connection goes down and comes back up, either the last modified date or the path changes: fix these and it should not re-hash.

ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Post by ivulfusbar » 2006-02-16 14:36

I have myself used hashed shares over SAMBA, and have no issues. The limit in my setting is the bandwidth and not the hashing. And we can't patch DC++ or SAMBA to help with bandwidth. ;))

But then again, i only had that setup for a couple of years. ;))
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

bastya_elvtars
Posts: 164
Joined: 2005-01-06 08:39
Location: HU
Contact:

Post by bastya_elvtars » 2006-02-16 18:32

I recall a samba setting which dramatically increased the performance.... will harvest from the mailinglist archives and post here.

-- // EDIT

Found. You have to mess with the socket options and check the media type of the interface Samba listens on, itshould be 100baseTX Full Duplex.

This is reported to work:

Code: Select all

socket options = TCP_NODELAY SO_RCVBUF=8192 SO_SNDBUF=8192
Hey you, / Don't help them to bury the light... / Don't give in / Without a fight. (Pink Floyd)

mrtoad
Posts: 3
Joined: 2004-10-25 08:30

Re: Hash managing not acceptable

Post by mrtoad » 2006-02-17 21:08

Hi

ullner wrote:
mrtoad wrote:This is really an issue for samba to fix, but they seem very content with samba performing at this horrid speed so I dont expect anything to happend there.
Right... So we should fix other's poor code? Or what? Why are you asking for an improvement of DC++ when the flaw is in Samba?

I explictily wrote that this is an issue with Samba and that the ppl working on Samba doesnt bother with this issue... (they havent for years!) Which is why it wouldnt do any good to post a message at their boards about samba being slow - I just mentioned this as an explanation why hashing can be slow. Of course I dont expect DC++ to fix this issue; however a solution that design around this issue would be neat.
ullner wrote:
mrtoad wrote: [...] Then you would manually update them [...]
Why don't you just set 'Auto refresh time' to 0? Then DC++ won't refresh the share when you've started DC++. (It will during startup.)


Thanks! I will try that.

mrtoad
Posts: 3
Joined: 2004-10-25 08:30

Post by mrtoad » 2006-02-17 21:22

ivulfusbar wrote:I have myself used hashed shares over SAMBA, and have no issues. The limit in my setting is the bandwidth and not the hashing. And we can't patch DC++ or SAMBA to help with bandwidth. ;))

But then again, i only had that setup for a couple of years. ;))


As I wrote I already have samba running on topspeed and Ive tried every samba fix there is I think... but still, top speed on samba is crawling compared to the speed on any other medium built since after 1850 :P

Thanks anyway thou :)

Locked