better support when working with segment-downloading clients

Use this forum to flesh out your feature request before you enter it in <a href="http://dcpp.net/bugzilla/">Bugzilla</a>.

Moderator: Moderators

Locked
mhanor
Posts: 14
Joined: 2006-01-15 09:30

better support when working with segment-downloading clients

Post by mhanor » 2006-01-15 10:20

The DC++'s behavior when uploading to clients which support segment-downloading is this: it displays that the client is downloading with higher speeds than in reality; also, DC++ doesn't set the progress bar's position accordingly to the position where the file is being resumed
I assume that these issues are DC++'s fault

Also, another feature that would be nice to be added to DC++ is some kind of throttling when encountering such behavior associated with segment-downloading from clients like StrongDC: canceling a transfer and resuming from a position, consecutive with the one where the client, which was downloading, canceled the transfer
A major improvement would be an improved behavior for such clients like StrongDC, which support segment downloading (see http://strongdc.berlios.de/forum/viewtopic.php?p=20163 ) , but it doesn't depend on DC++'s developers or DC++ itself

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 10:24

there's no real issue in DC++ neither in StrongDC++. The only issue is that DC++ logs every finish of $ADCSND command.

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 10:32

it's no true
DC++ does resets its upload status bar when a StrongDC-like client stops the transfer and resumes it
Also, I'm talking about DC++ v0.674 and StrongDC rev10 and derivates of it
these clients do not support the ADC protocol, I haven't tested clients or even hubs which support ADCSND

The StrongDC sends a $Cancel to the DC++ (like in the NMDC protocol) and the a $Send with the required position; this is repeated over and over again, by the StrongDC, when segment limits are reached. Or at least this is what I think it happens

the hub doesn't play any part in this cancel/send cycles, it only helps establish the connection between the clients, the fact is that if the hub fails and the DC++ doesn't disconnect the client which downloads from it (if it is configured to do so, there is an option for this), StrongDC manages to cancel and resume every time a segment is finished

bastya_elvtars
Posts: 164
Joined: 2005-01-06 08:39
Location: HU
Contact:

Post by bastya_elvtars » 2006-01-15 10:39

Sidenote: ADC was first implemented in 0.4033 if I remember well.
Hey you, / Don't help them to bury the light... / Don't give in / Without a fight. (Pink Floyd)

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 10:46

I don't know... but I'm sure the client is not using the ADC protocol
also, that command is a client-hub command, not a client-client command

bastya_elvtars
Posts: 164
Joined: 2005-01-06 08:39
Location: HU
Contact:

Post by bastya_elvtars » 2006-01-15 10:49

mhanor wrote:that command is a client-hub command, not a client-client command
Which? ADCSND, $Cancel, and $Send are client-to client AFAIK.
Hey you, / Don't help them to bury the light... / Don't give in / Without a fight. (Pink Floyd)

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 11:04

mhanor wrote:The StrongDC sends a $Cancel to the DC++ (like in the NMDC protocol) and the a $Send with the required position; this is repeated over and over again, by the StrongDC, when segment limits are reached. Or at least this is what I think it happens
StrongDC++ doesn't send $Cancel because it's not supported.

$ADCGET file TTH/xxxx 0 requested_bytes

a) DC++ does:
$ADCGET file TTH/xxxx 0 filesize

b) StrongDC++ does
$ADCGET file TTH/xxxx 0 chunksize

and every DC++ derived client logs and writes "Upload finished" after requested_bytes are transferred. It's the only problem.

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 11:06

yep... I was wrong, SND is client-client command, sorry for that
if StrongDC is using GET, maybe the "bytes" parameter of this command should be as big as it can possible be, not a fixed number, set by/equal with the segment size

And this means that DC++ is not behaving properly, not StrongDC
Am I getting this right?

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 11:12

both clients are behaving according to specification.

ADC protocol for SND command says: "The sender will keep on sending until <bytes> bytes of binary data have been sent, and then will put itself back to NORMAL state."

and when data have been sent, it means that uploading is finished.

EDIT:
solution would be that there would be some command $UploadFinished, which would StrongDC++ sned to client when it doesn't need any more chunk, but it's only waste of bandwidth
Last edited by Big Muscle on 2006-01-15 11:15, edited 1 time in total.

bastya_elvtars
Posts: 164
Joined: 2005-01-06 08:39
Location: HU
Contact:

Post by bastya_elvtars » 2006-01-15 11:14

Can CZDC++ detect partial uploads correctly? I am asking because they are logged separately.
Hey you, / Don't help them to bury the light... / Don't give in / Without a fight. (Pink Floyd)

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 11:22

of course, it can detect it, same as in StrongDC++, but it's detected according requested_bytes in $ADCGET.

bastya_elvtars
Posts: 164
Joined: 2005-01-06 08:39
Location: HU
Contact:

Post by bastya_elvtars » 2006-01-15 11:25

Big Muscle wrote:of course, it can detect it, same as in StrongDC++, but it's detected according requested_bytes in $ADCGET.
Does this mean any solution for DC++, or is that just a workaround? :P
Hey you, / Don't help them to bury the light... / Don't give in / Without a fight. (Pink Floyd)

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 11:30

it's not solution, because if I want from user A only the first chunk of file, then it won't be logged, because this chunk doesn't finish at the end of file.

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 11:36

wait... I don't mind that the requests are being logged in the Finished Uploads list as they are being logged
the thing that is kinda annoying is that the DC++ client interrupts the transfer at StrongDC's request... DC++ is doing what is supposed to do, but StrongDC could request more bytes that could fit in a segment, if it would benefit from it (if bytes are needed to complete the file, I'm talking about the bytes positioned after the position where DC++ would stop because StrongDC send a GET command with that fixed bytes number, which is the chunk size)

Example: let's assume the DC++ client has a file, which StrongDC makes the steps to aquire it... StrongDC calculates that it could download it as 3 chunks. Note that StrongDC doesn't find or the human behind StrongDC doesn't find another source from which to download it.
What is happening is this:
1) StrongDC request the fixed-chunk-size bytes, from position 0 (the begging of the file)
2) DC++ sends what has been requested (the bytes [0 - (chunk_size-1)] )
3) after requested bytes have been send, DC++ stops and StrongDC requests another fixed-size-chunk, from the position chunk_size
4) DC++ sends the requested bytes ([chunk_size, 2*chunk_size-1])
5) after requested bytes have been send, DC++ stops and StrongDC requests another fixed-size-chunk, from the position 2*chunk_size
6)DC++ sends the requested bytes ([2*chunk_size, 2*chunk_size+no_of_bytes_left])

More optimal would be that StrongDC would request file_size bytes. In the event that it finds another source and can do segment-downloading, StrongDC could choose a middle position in the not-downloaded domain, and start the download from the new source from that new position. StrongDC can cancel the transfer from the first source once that position has been reached and even terminate the connection if it determines that no segments of sufficient size can be downloaded from it.

Just see what behavior have ftp clients like Flashget when it comes to segment-downloading, even if sources are added long after the download started with the first source

Because StrongDC is doing what is doing, DC++ sends the file in pieces, and if the speed is high (several MB/s) they seem like burst transfers and I find it quite annoying... I would rather like to see continuous, uninterrupted trasnfers, if the conditions are right, and in most cases there are such conditions

Play with Flashget to see it's behavior... set it do download a file with one segment. After the download has started, add a new segment (with the same source, if the ftp server allows it), then a new segment... and so on... I don't know if the ftp protocol also allows n bytes to be requested, but the fact is that flashget and other clients (not necessarly ftp clients) accomplish the task of segment-downloading much better
Last edited by mhanor on 2006-01-15 11:54, edited 1 time in total.

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 11:51

you still don't understand the problem. No body doesn't interrupt the transfer. DC++ just sends the requested bytes.

And there's no possibility to cancel transfer except of disconnecting, which is risk of losing a precious slot.

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 11:58

hmm... you're saying that ADC doesn't support a cancel command... if there isn't another way to cancel the transfer, other than disconnecting, then there should be one :)
NMDC supports a command for cancelling a transfer

btw... I'm not saying that someone is disconnecting (you just keep repeating this)... I'm saying that DC++ is stopping when it shouldn't, and StrongDC could and should do something about it; cancelling a transfer is just a part of the problem, not the whole problem

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 12:08

NMDC supports $Cancel command, but it's not implemented in no client and it's no worth implementing it.
I'm saying that DC++ is stopping when it shouldn't
this isn't true. DC++ doesn't stop. Just imagine this:

You said me to give you 5 apples, so I give you 5 apples. And then you start saying that I stopped giving you apples, but I shouldn't :lol: :lol:

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 12:11

you have a point, but... why asking 5 apples when you know you need 10 apples and I'm the only vendor who sells apples around here

ullner
Forum Moderator
Posts: 333
Joined: 2004-09-10 11:00
Contact:

Post by ullner » 2006-01-15 12:14

bastya_elvtars wrote:Sidenote: ADC was first implemented in 0.4033 if I remember well.
.402 according to changelog.txt.

Big Muscle
Posts: 72
Joined: 2004-01-23 14:45

Post by Big Muscle » 2006-01-15 12:16

you don't know that no new vendor will appear there, so to avoid disconnecting you will request smaller amount.

mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 12:22

if the ADC protocol lacks a command for requesting a transfer cancel without distroying the network connection which has been established between 2 clients, then those responsible should add such a command to the protocol
I don't see any negative sides to adding such a command
from what you're telling me, I understand that the $Close command, from the NMDC protocol, is kinda despised... and I don't understand why...


mhanor
Posts: 14
Joined: 2006-01-15 09:30

Post by mhanor » 2006-01-15 12:37

yes... thank you.. I will read it
at least, it remains the issues regarding DC++'s transfer bar when uploading segments
it displays that the client is downloading with higher speeds than in reality; also, DC++ doesn't set the progress bar's position accordingly to the position where the file is being resumed
I assume that these issues are DC++'s fault

PseudonympH
Forum Moderator
Posts: 366
Joined: 2004-03-06 02:46

Post by PseudonympH » 2006-01-16 00:06

The only non-cosmetic issue is the case on very high-speed connections where the round-trip time for requests affects the throughput. Multisource implementations should increase the chunk size if transfers are completed faster than expected.

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Post by GargoyleMT » 2006-01-16 12:32

mhanor wrote:t remains the issues regarding DC++'s transfer bar when uploading segments
Yes. And the logs and finished uploads windows aren't too informative, or don't indicate that only a segment of the file was sent (since only a segment was requested).

Locked