Auto search and add new sources

Tr0n · Post by **Tr0n** » 2003-05-10 14:35

I've searched thru the whole forum, read all the posts and still couldn't find an answer to my question:

Why doesn't DC++ automatically search and add new sources to files in download queue?

New stuff usually comes in multiple archive files (.rar, .r01,..) right? So let's say you added one directory that contains such files. Now, in the download queue only that user is stored and usually waiting.

But if you do a manual search for alternative locations on one of the files, it usually finds more sources, which are then correctly added to that specific file in the download queue window.

The problem is, you have to do it manually for ALL files, if you want to find all the sources available to you (using multiple hubs of course).

Why doesn't DC++ do it automatically in reasonable intervals?

Can someone please explain?

DC++ v0.242
WinXP SP1a

Tr0n · Post by **Tr0n** » 2003-05-10 14:46

Oh, and I've got Automaticly search for alternative download locations turned on of course.

sarf · Post by **sarf** » 2003-05-10 15:44

Reasons:
#1 There are no other sources for that particular freeware program (or whatnot) that you are trying to download.
#2 You have too many other things in your download queue - DC++ auto-searches about once every minute - you do the math on how long it'll take for DC++ to iterate through your download queue.
#3 The people that do have the files you want do not use DC++, and since DC++ looks at the file extension and (maybe) determines .rar and .r0xy to be "Compressed" files it searches for them with a filetype set to, you guessed it, "Compressed". NM DC (or other clients) might not realize that .rar and .r0xy are "Compressed" filetypes, and so, regretfully, returns not a single search result.
#4 Something else

Hope this helps.

Sarf
---
If work were so wonderful, the rich would keep it all for themselves.

Tr0n · Post by **Tr0n** » 2003-05-10 15:46

None of the reasons apply, because if I do a manual search, sources are added just fine and the download begins. Didn't have any CRC error problems.

So as you see, this automated download thingy isn't working the way it should be.

Tr0n · Post by **Tr0n** » 2003-05-10 15:52

About your reason #2.

I left DC++ running for a whole day and night. Source count didn't change for a bit. Files that I didn't manually searched, still had only one user. After I triggered manual search, others were added.

#3. As I recall, alternative search only sets the file size of the file as "At least" and File type set at "Any". So I donÃ¢â‚¬â„¢t think, thatÃ¢â‚¬â„¢s the problem.

cologic · Post by **cologic** » 2003-05-10 16:01

The feature works beatifully for me, though I've heard many compaints about its ineffectiveness by others.

Tr0n · Post by **Tr0n** » 2003-05-10 16:04

cologic: So files, that only had one source "Waiting (user online)" after a while have more sources in your download queue window? For example "Waiting (10 of 10 users online)"?

cologic · Post by **cologic** » 2003-05-10 17:39

Usually by the time I notice, they're downloading already.

Tr0n · Post by **Tr0n** » 2003-05-11 00:17

Well, that's not the answer I'm looking for.

jbyrd · Post by **jbyrd** » 2003-05-11 02:59

I'm not quite sure, but I find mine works when the queue window is open, but not when it's not.

If that doesnt work, put the window up top (although it shouldnt matter).

-jbyrd

sarf · Post by **sarf** » 2003-05-11 03:55

Tr0n wrote:About your reason #2.

I left DC++ running for a whole day and night. Source count didn't change for a bit. Files that I didn't manually searched, still had only one user. After I triggered manual search, others were added.

This is the correct behaviour for the program. As I gaze into the beauteous code of DC++, I see that if a file has any source online, it will not be autosearched. I'll disable this check in DC++k (check out QueueManager.cpp, method onTimerMinute()).

Tr0n wrote:#3. As I recall, alternative search only sets the file size of the file as "At least" and File type set at "Any". So I donÃ¢â‚¬â„¢t think, thatÃ¢â‚¬â„¢s the problem.

Incorrect. This has been implemented since... well, since v0.23, at the very least. It is, however, a good idea - it only fails (part of the time) due to DC++ recognizing more filetypes than some other (names withheld to prevent public embarrasment) clients.

Note, however, that the "Search for alternates" menu choice does not care about file type.

I agree, jbyrd - whether the queue window is open or not should not matter... unless (lightbuld starts to shine above sarfs head... and goes out)... nope, wasn't that - I thought that when the download queue window was open it might be listening for search results. Well, it doesn't - at least not in v0.242. Oh well...

Sarf
---
Support the Math Illiteracy Tax - Buy Lottery Tickets

Tr0n · Post by **Tr0n** » 2003-05-11 04:46

This is the correct behaviour for the program. As I gaze into the beauteous code of DC++, I see that if a file has any source online, it will not be autosearched. I'll disable this check in DC++k (check out QueueManager.cpp, method onTimerMinute()).

I see. But it should check for alternative sources, even if it already has them. That way you can get files faster. You don't wait for one specific user. You check all of them and you start downloading from the user with a free slot.

DC++k?

sarf · Post by **sarf** » 2003-05-11 05:12

Tr0n wrote:[snip]I see. But it should check for alternative sources, even if it already has them. That way you can get files faster. You don't wait for one specific user. You check all of them and you start downloading from the user with a free slot.

Yes. This is what will happen (I hope) in DC++k, which is my own homegrown version of DC++. Just press the "www" button below my post(s) and download it. It should work, but as I didn't have time to test the latest version (deployed today)...

The reason for this behaviour is that the purpose of the "Automatically search for alternate sources" was meant to find sources for files which had no sources, not to speed up those who already had 'em.

Sarf
---
Only the insane have strength enough to prosper: only those that prosper truly judge what is sane.

Tr0n · Post by **Tr0n** » 2003-05-11 12:14

Cool! I'm going to try it out and report back.

Tr0n · Post by **Tr0n** » 2003-05-11 14:17

I suppose it kinda works. But the uploads don't start (always 0 kb/sec).

sarf · Post by **sarf** » 2003-05-12 09:22

Tr0n wrote:I suppose it kinda works. But the uploads don't start (always 0 kb/sec).

Argh! <sigh> That's what you get from thinking that the DC++ code base is the same version from version... oh well.

Going to kick some bug-behind... see you in a few hours (I hope).

Sarf
---
A good fight is like a stick of broccoli, but different.

sarf · Post by **sarf** » 2003-05-12 10:15

Problem fixed (for me, at least). Added some features you'll like, too, Tr0n.
New version should be up now.

Sarf
---
Software isn't released, it's allowed to escape.

Laikennus · Post by **Laikennus** » 2003-07-15 06:33

I found this topic with the same question of tron and... why the hell the programmers of dc++ don't implemment this too? the problem with dc++k is that many hubs don't allow it...

cyberal · Post by **cyberal** » 2003-07-15 06:37

I don't think the current version of k can be detected, not the same as the old cheat client..

Laikennus · Post by **Laikennus** » 2003-07-15 21:18

really? but i'm afraid... if I get permanently banned from a certain hub my hearth will stop for sure.
dc++k have all features of dc++? any disadvantage?

Well... but anyone knows why they don't put this feature in dc++?

Rodga · Post by **Rodga** » 2003-07-16 21:29

I've also experienced that the AutoSearch for new sources suxx. When I do it manually, it finds sources right away. This AutoSearch-thing does not work as it should!
Maybe the AutoSearch-thing does'nt add sources from the user with most slots per shared slots (open slots/shared slots). This AutoSearch should use common sense, so it adds sources from a user with 4/5 slots rather than from a user with 4/6. It should use math to choose the best download. Like 4/5 = 80 % (Best source) and 4/6 = 67 % (2nd best source).
If it thinks this way, then why does'nt it start the download(s) automatically (like it's ment to do)!?

jbyrd · Post by **jbyrd** » 2003-07-17 06:57

Because it is designed to search for sources only when all of your current sources are offline. Imagine the increase in searches if everyone automatically searched for downloads on every file that they was in the queue. Chaos would ensue.

Laikennus · Post by **Laikennus** » 2003-07-27 06:46

jbyrd wrote:Because it is designed to search for sources only when all of your current sources are offline. Imagine the increase in searches if everyone automatically searched for downloads on every file that they was in the queue. Chaos would ensue.

I think it could be solved increasing the time between searches...
better than almost nothing, because if a lot of users don't have that file that have only offline users in your queue, it's very dificult that someone connect to the hub with that file...(at last this happens to me)
But the case that 10 users have that file(but without free slots) and someone with that file connect to the hub is more frequent(because that file isn't too rare).

ivulfusbar · Post by **ivulfusbar** » 2003-07-27 09:17

the point is that autosearchs is not a good thing from a bandwidth point of view. Approx 95% of all searches sent to your client is probably an autosearch. I have always laughed on the fact that hubowners wants to see the number of hubs users are registered in, but not if the client has autosearch enabled or not. But i'm bias in my oppinion, and it can be usefull, specialy if you search for files that many have, then things works great. Some would argue that its usefull for the hard to get files also, and they might be correct. The question is, is the extra bandwidth used by clients hubs that autosearch uses worth it? I think a solution is to make dc++ more agressive, i.e., if autosearch finds a match in a directory add all alternative to all entries recursivly with the current match as root. This will ofcourse introduce more "missing files" reports between clients 2 clients, but what the heck, that will not strain the hubs or the clients.

if-i-get-bored-i-will-add-it-and-see-how-it-works-but-i-have-one-problem,-i-never-download-anymore-ly'ers ,))

mynickonlydivxfreak · Post by **mynickonlydivxfreak** » 2003-08-30 16:44

ivulfusbar wrote:the point is that autosearchs is not a good thing from a bandwidth point of view. Approx 95% of all searches sent to your client is probably an autosearch.

I would like to argue that the "killing the bandwidth" theory is an ignorant grasp in the air.

First of all, the reason the Hub's bandwidth suffer today is because people start multiple manual searches for files in their DL-queue and then leave them running until they return to their PCs. This could be hours in most cases and even longer in others. I even do it myself, why -you say? -Because I cannot sit in front of my PC all day long doing manual searches. -And I cannot just be satisfied with having 3-4 sources online for my file either, since it can take forever to wait for a free slot.
Since I'm downloading the newest movie releases as soon as the become availible on the net, it is essential for me, that I can keep adding sources as time passes. Very often I start out with one source and then keep adding as more and more people get the file. Quite a lot of people are doing the same.
"Why not wait a while before downloading the file, then there will be more sources", you might say. NO there wont, because the whole system is depending on more and more people downloading and sharing the file as fast as possible, spreading it like rings in the water.

So please stop making this ridiculous point about the bandwidth and let's get it implemented in the next release

.

Here's my theory

: I claim that extra bandwidth will actually be released, if we get these endless and irritating searches in a controled enviroment.
I suggest that an automated search is performed once every hour, for the files in your DL-queue. For the search not to go on and on (thereby putting the above comments to the ground), I suggest that the search will go through all the users online in the connected hubs, add non-queued sources and then disconnect. This, I think, will decrease overall search time in the DC network

.

This is of course not a scientific fact, but neither is the "killing bandwidth" theory, and I remain convinced that my arguements are a lot closer to the actual thruth than the ones I've heard against it.

For those thinking, that I am just another Kazaa-lover trying to change DC++, so it is easy enough for me to use, think again! - I hate Kazaa and all other faker/spyware supporting p2p programs. I have been connected to DC++ 24-7 for 8 months now and still going strong. But one should never settle for less, so I will keep expecting changes and improvements for as long as it is out there. It's all part of the sweet thing called progress.

To all you programmers working on this fantastic piece of software - keep up the spirit and the good work!

Post by **GargoyleMT** » 2003-08-31 10:33

mynickonlydivxfreak wrote:First of all, the reason the Hub's bandwidth suffer today is because people start multiple manual searches for files in their DL-queue and then leave them running until they return to their PCs.

What? DC++ searches once, not multiple-times. Search is not recurring. As that seems to be the basis of your argument, you really can't convince me with that.

Please respect fusbar, he's the reason why DC++ has had so many improvements to its search algorithm. He is in many hubs and has a rediculous amount of files - he is in the best position to see if DC++'s autosearches take up needless bandwidth and CPU time.

ivulfusbar · Post by **ivulfusbar** » 2003-08-31 10:46

mynickonlydivxfreak wrote: I would like to argue that the "killing the bandwidth" theory is an ignorant grasp in the air.

Shows how much you know about the problems for certain hubs.

mynickonlydivxfreak wrote: First of all, the reason the Hub's bandwidth suffer today is because people start multiple manual searches for files in their DL-queue and then leave them running until they return to their PCs. This could be hours in most cases and even longer in others. I even do it myself,

DC++ searches once and only once when you do a manual search. It will never search more times regardles if your searchframe is closed or not.

I don't see a point in answering or commeting anymore from this post as you have got everything wrong.

don't-ask-if-you-don't-want-to-hear-an-answer-ly'ers

mynickonlydivxfreak · Post by **mynickonlydivxfreak** » 2003-08-31 14:27

ivulfusbar wrote:DC++ searches once and only once when you do a manual search. It will never search more times regardles if your searchframe is closed or not.

Ok, then answer me this: If I start a search and wait for 5 minutes (I'm in 5 hubs with a total user-count around 8000), I normally get all the results the search can find at this moment. -But if I leave the search hanging and I come back 6 hours later, I can see that more sources have been found. These new results are from users, who have joined the hubs after I started the search or from users,who have added the file to their share. This is not occuring once in a while, but every day when I and others are using the system. I have also encountered hubs, that will ban users, who leave their search hanging.
Please enlighten me, how can all of this be, if the search only goes through the online users once and then stop completely?

Btw I really ment no disrespect, I was merely trying to provoke the discussion, since very few have pro-arguements about this topic.

Also, I still believe that it should and can be implemented in DC++, after all, lots of much larger online databases seems to have managed to find a way to make it work.

And last, even if it means more presure on the hub's bandwidth, so what? It only means that you need to have a connection type, that matches the amount of users, that you want to join your hub. Today it is the hub-owners making all the requirements of the users, why should requirements not be made indirectly the other way around too. After all, if there were no users, there would be no use for hub-owners either, so they need us just as much as we need them

Post by **GargoyleMT** » 2003-09-02 10:34

mynickonlydivxfreak wrote:Ok, then answer me this: If I start a search and wait for 5 minutes (I'm in 5 hubs with a total user-count around 8000), I normally get all the results the search can find at this moment. -But if I leave the search hanging and I come back 6 hours later, I can see that more sources have been found. These new results are from users, who have joined the hubs after I started the search or from users,who have added the file to their share. This is not occuring once in a while, but every day when I and others are using the system.

You're free to trace the code yourself. I'm very convinced that standard DC++ doesn't search but once when you initiate a search though the GUI.

SearchFrm::onEnter
SearchManager::search
ClilentManager::search
this iterates through the connected clients
Client::search

The "$Search" string only appears in the Client::search function, and I don't see it being called anywhere based on an onAction of a TimerManager listener.

Some DC++ mod may have changed this, but it's not the default behavior of DC++... unless I can be convinced otherwise.

WOW I was so quick to engage in pedantry that I didn't notice that you said sources. Of course that's the expected behavior, you must have "auto-search for alternate sources" turned on under Advanced preferences. However, that's not quite exactly what you said initially ("hanging open searches")....

cyberal · Post by **cyberal** » 2003-09-04 12:51

my thoughts..

mynickonlydivxfreak> the new search results are probably from clients that responded to the search request after you left your computer.

Your claim that they are new users is simply wrong and somewhat odd btw, how could you possibly know that they were not in the hub earlier? you keep track of all the users?
-------------

the "automaticly search for alternates" is a big problem when the user i passive. It drasticly lowers the amount of users the hub can handle, IMO this option should be disabled for ppl in passive mode!

cyberal · Post by **cyberal** » 2003-09-04 13:00

mynickonlydivxfreak wrote:And last, even if it means more presure on the hub's bandwidth, so what? It only means that you need to have a connection type, that matches the amount of users, that you want to join your hub. Today it is the hub-owners making all the requirements of the users, why should requirements not be made indirectly the other way around too. After all, if there were no users, there would be no use for hub-owners either, so they need us just as much as we need them

this is just plain stupid (sorry, but it is)

TheNOP · Post by **TheNOP** » 2003-09-05 06:22

cyberal wrote:
mynickonlydivxfreak wrote:And last, even if it means more presure on the hub's bandwidth, so what? It only means that you need to have a connection type, that matches the amount of users, that you want to join your hub. Today it is the hub-owners making all the requirements of the users, why should requirements not be made indirectly the other way around too. After all, if there were no users, there would be no use for hub-owners either, so they need us just as much as we need them
this is just plain stupid (sorry, but it is)

i agree with you cyberal.

i run server because i hate paying for somethings and only use a fraction of it....
while i sleep, with no server, i get the feeling i pay for notings, so i share my bandwidth with others.(primarely for my familly and friends)
i have to thanks the users for taking that feeling(not that it would be difficule to live with it), but if they want to use my ressource they will/must follow the rules in place or go elsewhere.
no one will tell me what to do with my ressource, certainly not strangers.
note that i almost never dl or upl from dc or other P2P, so don't args that it brings me lots of stuff.
i don't think that running a mail server brings me stuff neder, else then being able to always keep the same e-mail address and filter spam mail at the source.
from your point of view mynickonlydivxfreak i should be gratefull the others use my servers ?

DC++

Auto search and add new sources

Auto search and add new sources

Re: Auto search and add new sources

maybe...

Re: How does AutoSearch think? What can be improved?