0.689 is slow to read large lists [1TB] from 0.647 and hangs

Know of something that might be useful to the DC community? Post it here! (Still, no advertising)

Moderator: Moderators

Locked
optionwizz
Posts: 6
Joined: 2006-02-03 16:01

0.689 is slow to read large lists [1TB] from 0.647 and hangs

Post by optionwizz » 2006-05-03 07:25

0.689 is slow to read large lists [1TB] from 0.647 and seems to hang.
.689 reads the list in very slowly and partially, showing 80/90 gig of a 1 TB share after 10's of minutes, and will not respond to browsing folders, but a search of the 1TB .647 for a known file will get it.

I have a hunch from Ullner's blog that its tied to numbers of items in memory or the new way of searching lists.

Smaller 0.647 shares, 150 to 200 gig respond well in active or passive modes, firewall is sygate and is off on winXP & sp2 + updates, no firewall. no router.
Loading a dc++ 0.647 [DCDraja version] at the same time, works on all lists. ports are set TCP, UDP = 4411, logs and settings available.

in private hub.
Jim :twisted:

ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Post by ivulfusbar » 2006-05-03 07:45

I would not expect DC++ to handle 1TiB filelists. Most computers doesn't have 1TiB of memory to store the information on.

If you on the other hand mean someone sharing 1TiB, then that information is rather useless. What is interesting is the number of items shared, directories and or files. Sharing 250 4 GiB iso-files (from my wedding) would for example be opened in a snapp.

Myself i have no issue with browsing larger filelists. Can you say anything about the number of files of the filelist you are browsing?
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

optionwizz
Posts: 6
Joined: 2006-02-03 16:01

file quanties being gathered

Post by optionwizz » 2006-05-03 15:47

I have asked the holders of the larger shares to gather the quanties of files in their shares, 1 is over 100,000, mine is 600 gig and is over 125,000 files, [ a full win 2k or win2003 700 gig cd is 6000 to 7000 files]

will post later or i can make you a guest login on a private hub.
Jim

TheParanoidOne
Forum Moderator
Posts: 1420
Joined: 2003-04-22 14:37

Re: file quanties being gathered

Post by TheParanoidOne » 2006-05-03 15:53

optionwizz wrote:[ a full win 2k or win2003 700 gig cd is 6000 to 7000 files]
I assume you mean megabytes? You seem be throwing values and units all over the place ...
The world is coming to an end. Please log off.

DC++ Guide | Words

bastya_elvtars
Posts: 164
Joined: 2005-01-06 08:39
Location: HU
Contact:

Post by bastya_elvtars » 2006-05-03 16:06

1TB must be the sharesize that the filelist belongs to.
Hey you, / Don't help them to bury the light... / Don't give in / Without a fight. (Pink Floyd)

optionwizz
Posts: 6
Joined: 2006-02-03 16:01

users shares files [folders soon]

Post by optionwizz » 2006-05-03 18:10

users shares files client os ram [folders soon]
1 2.66 TB 161,161 .674 XP-sp2 1G
2 1.33 TB 173,413 .674 xp-sp2 1G
3 1.07 TB 191,197 .687 XP-SP2 1G
4 751 GIG 145,900 .674 XP-SP2 1G
5 627 GIG 24,137 .674 XP-SP2 1G
6 622 GIG 98,202 .674 XP-XP2 1G
7 589 GIG 165,765 .689 XP-SP2 1.5G 14,000+
great file details available
8 549 GIG 36,447 .689 XP-SP2 1G
9 515 GIG 3,452 .674 XP-SP2 1G
10 485 GIG .305 RESPONSE TOMORROW

all on P4-2.6 Giga to 3.4 Giga :roll:
hub 43 users, 12+TB.

A user with 106 gig 18821 files just went back to .687

SOFTWARE= karens disk slack checker 2.5.1 and Tree size pro 3.33
You are welcome to pop into the hub to verify these.
jim

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Re: 0.689 is slow to read large lists [1TB] from 0.647 and h

Post by GargoyleMT » 2006-05-05 16:52

optionwizz wrote:.689 reads the list in very slowly and partially, showing 80/90 gig of a 1 TB share after 10's of minutes, and will not respond to browsing folders, but a search of the 1TB .647 for a known file will get it.
Can you explain what you mean? I can't tell what you're describing.

optionwizz
Posts: 6
Joined: 2006-02-03 16:01

This seems to describe the problem in .689 and the dcFul 689

Post by optionwizz » 2006-05-14 02:42

from the the changelog on the dcFUL site
this seems to get worse with bigger file lists
http://www.fuldc.net/download.php


his quote
Added: threaded file list loading, should stop the client from hanging while opening large lists (thanks xayide)

Thanks
jim

optionwizz
Posts: 6
Joined: 2006-02-03 16:01

Help from Trem on list loading in 0.689FUL

Post by optionwizz » 2006-05-14 05:58

02:46] <packrat> so my point is the guys with 100,000 shared files, and up, seem to be the worst.
[03:42] <Trem> do you try to change folder while the status bar still displays please wait loading file list? then it'll crash or display weird folder structures
[03:42] <Trem> it'll take up to a few minutes before the list is fully displayed on large shares
[04:15] <packrat> thanks
[04:16] <packrat> but it is 15 minutes +, i will test on the 1.9TB share. 380,000 files on a client 0.674.

after 55 minutes list still not displaying.

Same list on dc++ 0.674 took 10 minutes

Similar very slow results on normal dc++ 0.689

hope that helps.
jim :roll:

optionwizz
Posts: 6
Joined: 2006-02-03 16:01

DZDC 0.666[N] has fixed the problem

Post by optionwizz » 2006-05-16 09:18

DZDC 0.666[N] has fixed the problems and id very quick.
The color addons are nice too.
jim :P

Locked