Slot notification inefficent

Technical discussion about the NMDC and <a href="http://dcpp.net/ADC.html">ADC</A> protocol. The NMDC protocol is documented in the <a href="http://dcpp.net/wiki/">Wiki</a>, so feel free to refer to it.

Moderator: Moderators

Locked
Nocturnal
Posts: 6
Joined: 2003-04-09 06:53
Location: Wellington, New Zealand

Slot notification inefficent

Post by Nocturnal » 2003-04-09 08:29

Why is it connecting to sources every few seconds TCP to check for open slots, shouldnt the client just jump on a remote queue and then be notified by UDP when they reach the top of the queue ie when a slot is available?

Run netstat and you see multiple TCP connections to the same host all in time wait status.

andlju
Posts: 27
Joined: 2003-02-28 12:58
Location: Stockholm, Sweden
Contact:

Post by andlju » 2003-04-09 08:53

A similar idea has been proposed here:

http://dcplusplus.sourceforge.net/forum ... php?t=1381

I'd really like this feature too, it would make the DC network just a bit more fair. It should, in my opinion, be implemented the way "sarf" proposes, using "reverse connecting".

/Anders

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Re: Slot notification inefficent

Post by GargoyleMT » 2003-04-09 10:45

Nocturnal wrote:Why is it connecting to sources every few seconds


Note, in DC++, "few" is either 50 or 60 seconds. That's how long it takes before DC++ decides to check for open slots.

See also "upload queueing code" by cologic in this forum. It's not full featured, but it is a good starting point.

Upload queues are being worked on and thought of. I don't think a separate mechanism needs to be invented to tell someone that they're next in line, nor does the slot checking need to be done more infrequently.

You realize that right now, DC++ doesn't queue uploads at all, it's just luck of the draw on who gets open slots, right?

Sapporo
Posts: 36
Joined: 2003-02-09 23:10
Location: AZ, USA

Re: Slot notification inefficent

Post by Sapporo » 2003-04-09 11:12

GargoyleMT wrote:You realize that right now, DC++ doesn't queue uploads at all, it's just luck of the draw on who gets open slots, right?


Yeah, sometimes I'm really lucky and other times, well I'm not. It's just another DirectConnect protocol annoyance.

Nocturnal
Posts: 6
Joined: 2003-04-09 06:53
Location: Wellington, New Zealand

Emule queue system

Post by Nocturnal » 2003-04-10 00:43

Emule has quite an inteligent queuing system which uses the idea of credits whereby each client is rewarded for uploading by advancing in the time based queue more quickly. Each client has a unique identifier and maintains a list of other clients identifiers and how many Mb they have uploaded. Eg Client A uploads so many Mb to client B, client B records client A's id and how many Mb then next time client A wants to download from client B a time based multiplier (exponential decay relationship to Mb) is calculated eg 2 meaning every minute on client A's queue will count as two compared to a fair queue. The system is resistant to queue cheating as your Mb data is stored on other clients that you have uploaded to.

Also I think some hashing system and segmented downloading system is needed. The emule uses md4 hash calculated over blocks of 9500kb then a master hash calculated over the whole file to verify the integrity of a completed download. As each 9500kb block is downloaded and verified it is made available to share to other clients negating the need to download the whole file before it can be reshared. If the reported hash does not match the calculated hash the block is discarded and redownloaded. (They do have an option called inteligent corruption handing that is supposed to reduce the need to redownload the whole block if possible - I have not gone over the code to determine how this works). Hashing is a necessity because a corruption in someones download can be duplicated multiple times to everyone who uses them as a source, wasting much bandwidth and time.

The source is available at http://emule-project.net

Nocturnal
Posts: 6
Joined: 2003-04-09 06:53
Location: Wellington, New Zealand

Corruption

Post by Nocturnal » 2003-04-10 00:47

PS Also the file may download fine but may be corrupted locally ie on the filesystem or local lan. Therefore any hashing system should periodically rehash a percentage of files shared to revalidate the integrity of the local copy.

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2003-04-10 06:35

This thread documents some user rating ideas people here have had pertaining to DC++; in a couple of recent threads now I've linked to it...[/url]

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Post by GargoyleMT » 2003-04-10 06:49

If you're a fan of eMule, you'll see it mentioned a lot in some of my past feature suggestions. Sandos also mentions it, and cologic has looked at it for his planned integration of tiger tree hashes into DC++.

Locked