Persistent download history

Problems compiling? Don't understand the source code? Don't know how to code your feature? Post here.

Moderator: Moderators

Locked
Sunnyboy
Posts: 6
Joined: 2005-12-25 09:12

Persistent download history

Post by Sunnyboy » 2005-12-25 09:27

Hi all!

I want to implement a download history which is stored on disk for future use. I do not want to download some files twice. So files will be red in download queue when you add a duplicate.

So the question is the following. Is it enough to store only TTH root value to identify file? Or may be I need to store TTH + file size? As I understand two or more files can have a the same TTH root value.

Thanx.

ullner
Forum Moderator
Posts: 333
Joined: 2004-09-10 11:00
Contact:

Re: Persistent download history

Post by ullner » 2005-12-25 10:00

Sunnyboy wrote:Is it enough to store only TTH root value to identify file?
Yes.
Sunnyboy wrote:Or may be I need to store TTH + file size? As I understand two or more files can have a the same TTH root value.
No. Store "abc" in filea.txt and "def" in fileb.txt. They will have the same size but NOT the same TTH. With TTH, the name and file size is disregarded.

Sunnyboy
Posts: 6
Joined: 2005-12-25 09:12

Re: Persistent download history

Post by Sunnyboy » 2005-12-25 11:47

ullner wrote:No. Store "abc" in filea.txt and "def" in fileb.txt. They will have the same size but NOT the same TTH. With TTH, the name and file size is disregarded.
Thanx ullner.

But can be the situation when two files with different size have the same TTH? If yes, I think it will be more effective to store TTH + file size to identify files in future. May be it is redundant. I'm not sure.

joakim_tosteberg
Forum Moderator
Posts: 587
Joined: 2003-05-07 02:38
Location: Sweden, Linkoping

Re: Persistent download history

Post by joakim_tosteberg » 2005-12-25 12:45

Sunnyboy wrote:
ullner wrote:No. Store "abc" in filea.txt and "def" in fileb.txt. They will have the same size but NOT the same TTH. With TTH, the name and file size is disregarded.
Thanx ullner.

But can be the situation when two files with different size have the same TTH? If yes, I think it will be more effective to store TTH + file size to identify files in future. May be it is redundant. I'm not sure.
What I am aware of have no TTH collisions been detected so far and if a collision would occur. I would say that it did would it be time to find a new hashing technique to use.

Sunnyboy
Posts: 6
Joined: 2005-12-25 09:12

Post by Sunnyboy » 2005-12-25 17:03

Ok. Now let me ask in this form: How can I implement the reliable download history? :lol: Please give some advices.

Thanx.

ullner
Forum Moderator
Posts: 333
Joined: 2004-09-10 11:00
Contact:

Post by ullner » 2005-12-25 17:17

Use the download log that DC++ can create.

Sunnyboy
Posts: 6
Joined: 2005-12-25 09:12

Post by Sunnyboy » 2005-12-25 18:16

ullner wrote:Use the download log that DC++ can create.
DC++ logs filename, size, speed etc., but not TTH. So it is not a solution :(

Todi
Forum Moderator
Posts: 699
Joined: 2003-03-04 12:16
Contact:

Post by Todi » 2005-12-25 18:54

Sunnyboy wrote:DC++ logs filename, size, speed etc., but not TTH. So it is not a solution :(
DC++ Helpfile wrote:Download and Upload Log Format
%[user] - User name
%[userip] - User's IP address
%[hub] - Hub name
%[hubip] - Hub's IP address
%[size] - File Size
%[sizeshort] - File size, shortened and including units
%[chunksize] - Size uploaded this session
%[chunksizeshort] - Size uploaded this session, short and including units
%[actualsize] - Actual uploaded bytes, affected by compression
%[actualsizeshort] - Actual uploaded bytes, short and including units
%[speed] - Speed of the transfer
%[time] - Elapsed time of the transfer
%[sfv] - Whether the file was checked against a SFV file (0 = no, 1 = yes).
%[tth] - Base32 representation of the tiger tree root hash
Only for Download log: %[target] - Local path and filename
Only for Upload log: %[source] - Local path for the upload.
Default download log format: %Y-%m-%d %H:%M: %[target] downloaded from %[user], %[size] (%[chunksize]), %[speed], %[time]
Default upload log format: %Y-%m-%d %H:%M: %[source] uploaded to %[user], %[size] (%[chunksize]), %[speed], %[time]
Press F1 in DC++ to get further information on how to use DC++ more efficently.

ullner
Forum Moderator
Posts: 333
Joined: 2004-09-10 11:00
Contact:

Post by ullner » 2005-12-25 21:23

Though, If I remember correctly, %[tth] for uploads doesn't work (there's no code that support it).

cologic
Programmer
Posts: 337
Joined: 2003-01-06 13:32
Contact:

Post by cologic » 2005-12-25 22:13

For the last year (since some 0.67x version, I guess),

Code: Select all

		if(u->getTTH() != NULL) {
			params["tth"] = u->getTTH()->toBase32();
		}
has existed in UploadManager.cpp. I haven't tested it to work, nor have I ever intentionally used it, but support code does exist.

Sunnyboy
Posts: 6
Joined: 2005-12-25 09:12

Post by Sunnyboy » 2005-12-26 04:28

Thanx for all.

I already wrote a class(singleton) History which listens events from DownloadManager and writes TTH of all completed files in hash_set. Then in QueueManager I check new items, and if their TTH already exist in History, I mark them with special flag. This flag is checked in queue list to determine color of item.

So the original question was - Can I rely on TTH? Do I have to log not only TTH, but may be size of file or some additional checksum to reduce the risk of mistake? I know that TTH can lead to collision. What is the best set of information do I have to log?

Thanx.

ullner
Forum Moderator
Posts: 333
Joined: 2004-09-10 11:00
Contact:

Post by ullner » 2005-12-26 04:56

Sunnyboy wrote:So the original question was - Can I rely on TTH? Do I have to log not only TTH, but may be size of file or some additional checksum to reduce the risk of mistake?
Yes, you can rely on TTH. There is no use of you using the file type to ensure file indentity.
Sunnyboy wrote:I know that TTH can lead to collision.
Well, it could but there have been no records of a collision concerning TTH.
cologic wrote:For the last year (since some 0.67x version, I guess),

Code: Select all

      if(u->getTTH() != NULL) {
         params["tth"] = u->getTTH()->toBase32();
      }
has existed in UploadManager.cpp. I haven't tested it to work, nor have I ever intentionally used it, but support code does exist.
Yes, you could use %[tth]. Though, when a upload is complete (or begin, I don't remember which), no TTH flag is set so you don't get the TTH of the upload. This is what I meant by 'no code support'. Aswell, if I remember correctly, DC++ throws an exception from the above code because there's no TTH to convert to base32.

Sunnyboy
Posts: 6
Joined: 2005-12-25 09:12

Post by Sunnyboy » 2005-12-26 16:06

Thanx guys!

Locked