Time to write a complete RFC.

Technical discussion about the NMDC and <a href="http://dcpp.net/ADC.html">ADC</A> protocol. The NMDC protocol is documented in the <a href="http://dcpp.net/wiki/">Wiki</a>, so feel free to refer to it.

Moderator: Moderators

Locked
ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Time to write a complete RFC.

Post by ivulfusbar » 2003-01-14 04:50

I think its time for us to combine our knolwedge and write a correct version of the protocol. All i have seen so far have some misstakes or lack some information.

So what do you all think?

it's-time-to-join-things-like-$QuickList-$Support-and-other-things-into-a-unified-protocol-ly'ers ;))
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

aDe
Forum Moderator
Posts: 138
Joined: 2003-01-07 09:14
Location: SE
Contact:

Post by aDe » 2003-01-14 07:12

I have been thinking about this for a while and I might write one.. I'll see if I get the time!

sandos
Posts: 186
Joined: 2003-01-05 10:16
Contact:

Post by sandos » 2003-01-14 08:20

Someone could put a wiki up, letting everybody write to the rfc. Once in a while, someone then copies that to a "official" read-only rfc.

Sedulus
Forum Moderator
Posts: 687
Joined: 2003-01-04 09:32
Contact:

Post by Sedulus » 2003-01-14 09:00

shouldn't the $ForceMove be fixed?

the client can now decide to stay online, which is bad

a quick fix would be to always disconnect the user after n seconds...
http://dc.selwerd.nl/hublist.xml.bz2
http://www.b.ali.btinternet.co.uk/DCPlusPlus/index.html (TheParanoidOne's DC++ Guide)
http://www.dslreports.com/faq/dc (BSOD2600's Direct Connect FAQ)

ender
Posts: 224
Joined: 2003-01-03 17:47

Post by ender » 2003-01-14 14:31

Sedulus wrote:shouldn't the $ForceMove be fixed?
I think that the only reason why $ForceMove doesn't disconnect the client (when using NMDC hub) is because Mr. Hess didn't know how to flush the socket before disconnecting it...

TasMan
Posts: 196
Joined: 2003-01-03 08:31
Location: Canada
Contact:

Post by TasMan » 2003-01-14 15:04

I'd be willing to write a "complete" protocol. I've registered a new project on SF, and am planning a hub soft, a client and a bot. I'd obviously like a complete protocol to do this right!

It's been "started" (ok it was a 30 minute job so there are probably a few mistakes - and the descriptions aren't very descriptive/helpful :p ), with only client <-> hub protocol.

http://sourceforge.net/docman/display_d ... p_id=71384

Gumboot
Posts: 22
Joined: 2003-01-15 20:08
Location: New Zealand

Post by Gumboot » 2003-01-15 23:48

FYI, you seem to be missing a close bracket ')' in the $SR description.

ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Post by ivulfusbar » 2003-01-16 02:21

these short versions is not what i had in mind where you only count different commands and how they are sent.

I mean a real protocol describing all the structure.
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

TasMan
Posts: 196
Joined: 2003-01-03 08:31
Location: Canada
Contact:

Post by TasMan » 2003-01-16 05:50

I'd be willing to do that too....though your first message wasn't too descriptive so I figured I do whatever :P

Thanks Gumboot

Meltingfire
Posts: 9
Joined: 2003-01-06 07:23
Location: Malmo, Sweden
Contact:

Post by Meltingfire » 2003-01-16 12:55

Im willing to host a page about the protocol on the comming DC portal.. just tell me how you want it to look and what you will be able to do, and i can code it..
Image

Neg
Posts: 20
Joined: 2003-01-19 07:05

Post by Neg » 2003-01-20 11:39

Hi

The ide of a RFC document is good but i think we need to make some major chnages in the protocole. Insted of adding varius hacks to the NMDC protocole i think we shulde trhow away NMDC compability and rewrite the protocole to support many new feats.

Im no big time user of DC++ i use my own home coded client that is a hybrid that requers both a *nix server and a win based client software. So if i make a change in the protocol as i feel there is a huge need to do in many departments it will have no effect in the community so i feel a need to discuss my thoughts whit far more experienced developers in many areas i think more work is needed.

I will bring up some ares i feel more work need to be done in and if you have any feedback please tell me because thats why im writing this.

And i see a need to apply some Public/Private Key encryptin to both the Clinet/Hub and Client/Client protocols.

Optianly a fast data compresor for the client to client file transfers to reduce data needed to be transferd, maybe expand the zlib iniative that DC++ have taken on the hub lists?

Automatic some sort of CRC check of all files in both search and upload stage.

I can think of some more minior protocol changes to but not so drastic as these ones. I think the DC community is great but its starting to become a chaos of difrent protocol hacks that is badly documented so i think some project that have a large influens in the community need to sit down and write a RFC for a DC-2 protocol or sutch.

All also think that multisearch feats shulde be removed because it only makes the protocol more complex and create almoste identical duplicates of the samme command witch adds to bandwith use. And the commands $Hello $getINFO, $GetNickList, $NickList and $OpList shulde be removed from the protocol and soly use a modifyed version of $MyInfo and $Quit.

I have a beginning to a RFC of a new protocol i started when i was planing some improvments to my own DC client witch i can post here if anyone is intrested to pick it up and rework it into someting usefull.

I also think the ides of more chat support is insane it will just add more load to the servers if you want to chat get a chatclinet that dose the work mucth better.

Please let me know what you think of this.

sandos
Posts: 186
Joined: 2003-01-05 10:16
Contact:

Post by sandos » 2003-01-20 12:11

Neg wrote:I can think of some more minior protocol changes to but not so drastic as these ones. I think the DC community is great but its starting to become a chaos of difrent protocol hacks that is badly documented so i think some project that have a large influens in the community need to sit down and write a RFC for a DC-2 protocol or sutch.
"Large influence" here must be DC++. Not a rfc, not this forum, just the latest implementation of DC++. Whatever is in it, will become a standard IMHO.

On the hubside its not as clear though, lots of good hubs out there and no one clear winner (winner as in popular) as I can see.

Btw, code is the best documentation, you dont have to explicitly write the docs, and its always current. :) (Yeah I know its hard to read)

Neg
Posts: 20
Joined: 2003-01-19 07:05

Post by Neg » 2003-01-20 13:04

Btw, code is the best documentation, you dont have to explicitly write the docs, and its always current. (Yeah I know its hard to read)
Code is not hard to read but its generic bad to use it as development documentation when developing new clients. If the code is bad commented the 2nd developer might have to guess what some commands dose and what the expected effect of a command is.

And if we are to develop a new protocol or even addons to the exsisting one i think there need to be a protocol define so both client and hub software developers can add there thughts to it befor it becomes to working document of the protocol. And so both client and hub can be codeveloped at tow diffrent sites and crews and still work together.

yilard
Posts: 66
Joined: 2003-01-11 06:04
Location: Slovakia

Post by yilard » 2003-01-21 05:15

And i see a need to apply some Public/Private Key encryptin to both the Clinet/Hub and Client/Client protocols.
Why the hell, do you need Public/Private Key encryption??? At first, such encryption is very resource demanding. So Symmetric key encryption is usually used to transfer great amounts of data.

But anyway, I think encryption is of no use in dc++. Anyone can read you filelist so why do you want to encrypt anything?
Optianly a fast data compresor for the client to client file transfers to reduce data needed to be transferd, maybe expand the zlib iniative that DC++ have taken on the hub lists?
Compressing files wouldn't be great benefit, because people tend to share files, that can't be compressed further (avi, mp3, rar, etc.). It would only add additional CPU load to both clients.
Automatic some sort of CRC check of all files in both search and upload stage.
SFV checking tries to do something similar, but I didn't felt need for such feature yet, because there is much greater possibility that user is sharing already corrupted file, than that it becomes corrupted during transfers.
I can think of some more minior protocol changes to but not so drastic as these ones. I think the DC community is great but its starting to become a chaos of difrent protocol hacks that is badly documented so i think some project that have a large influens in the community need to sit down and write a RFC for a DC-2 protocol or sutch.
I think it is a good idea to document current state of DC protocol. But I don't think that any effort should be put into introducing new features. I think it is pretty well functional and It would be better to finish client and to overcome its performance shortcomings.
I also think the ides of more chat support is insane it will just add more load to the servers if you want to chat get a chatclinet that dose the work mucth better.
Hm, maybe chat doesn't add so much load on hubs. On many hubs, there is very little chat and they experience performance problems. I think search requests are what put load on them. The builtin chat is what gives some feel of community and sharing. (in contrast Napster was, and kazaa is just about leeching).
In the age of super-boredom/hype and mediocrity/celebrate relentlessness/menace to society --KMFDM

FRANKE
Posts: 26
Joined: 2003-01-03 11:29
Location: Denmark
Contact:

Post by FRANKE » 2003-01-21 06:15

yilard wrote:Hm, maybe chat doesn't add so much load on hubs. On many hubs, there is very little chat and they experience performance problems. I think search requests are what put load on them. The builtin chat is what gives some feel of community and sharing. (in contrast Napster was, and kazaa is just about leeching).
Amen
sandos wrote:Btw, code is the best documentation, you dont have to explicitly write the docs, and its always current. :) (Yeah I know its hard to read)
I might be wrong, but I think code is the worst kind of documentation, especially because the protocol should be langue independent.
For the newest release of MulTiBoT visit www.cwain.dk

FRANKE
Posts: 26
Joined: 2003-01-03 11:29
Location: Denmark
Contact:

Post by FRANKE » 2003-01-21 06:17

langue = language :)

You cannot edit your posts in this forum << why ?
For the newest release of MulTiBoT visit www.cwain.dk

sandos
Posts: 186
Joined: 2003-01-05 10:16
Contact:

Post by sandos » 2003-01-21 06:43

FRANKE wrote:
yilard wrote:Hm, maybe chat doesn't add so much load on hubs. On many hubs, there is very little chat and they experience performance problems. I think search requests are what put load on them. The builtin chat is what gives some feel of community and sharing. (in contrast Napster was, and kazaa is just about leeching).
Amen
sandos wrote:Btw, code is the best documentation, you dont have to explicitly write the docs, and its always current. :) (Yeah I know its hard to read)
I might be wrong, but I think code is the worst kind of documentation, especially because the protocol should be langue independent.
I wasnt seriously meaning that we should rely on source as documentation, documentation is very nice to have. I do however feel that there is lots of talk and not to much being done about things. I know I dont code anything, which I actually should since I feel the need for certain things.

Neg
Posts: 20
Joined: 2003-01-19 07:05

Post by Neg » 2003-01-21 12:16

Why the hell, do you need Public/Private Key encryption??? At first, such encryption is very resource demanding. So Symmetric key encryption is usually used to transfer great amounts of data.

But anyway, I think encryption is of no use in dc++. Anyone can read you filelist so why do you want to encrypt anything?
Public/Privte key eles they can log the key and the encrypion is totaly worthles. If we cryptate it we can make secure members only hubs and no one can see whats going on there. And even on puplic hubs it will add to securety. Who ever is looking for what you are sharing cant just tap the connectin it must acctulay connect to the hub puts more effort in to it. Also the file transfers can be made using some low CPU consuming non Public/Private key encryption when the key can be exchange in a secure way.
Compressing files wouldn't be great benefit, because people tend to share files, that can't be compressed further (avi, mp3, rar, etc.). It would only add additional CPU load to both clients.
Well there is many uncompresd types to eg. iso witch both is large and more than offen can be compresed quiet a bit. So maybe we shulde add compresion support and then its up to the tow clients to agree on compresing or non compresed transfers.
SFV checking tries to do something similar, but I didn't felt need for such feature yet, because there is much greater possibility that user is sharing already corrupted file, than that it becomes corrupted during transfers.
I did not meant this feat to be applayed as a check after the file transfer. The CRC shulde be sent to to downloding client befor it starts downloading and if it is a resume matched agains a stored CRC in the queue list to check that you dont resume from a bad copy.
Hm, maybe chat doesn't add so much load on hubs. On many hubs, there is very little chat and they experience performance problems. I think search requests are what put load on them. The builtin chat is what gives some feel of community and sharing. (in contrast Napster was, and kazaa is just about leeching).
I thnk the chat as it is today is good but what i ment was there is no need to extend it to use special chat channels and sutch in a IRC fashion. I have seen ides for this somewhere and i just wanted to point out that i dont think its a good ide.

yilard
Posts: 66
Joined: 2003-01-11 06:04
Location: Slovakia

Post by yilard » 2003-01-21 13:07

Public/Privte key eles they can log the key and the encrypion is totaly worthles. If we cryptate it we can make secure members only hubs and no one can see whats going on there. And even on puplic hubs it will add to securety. Who ever is looking for what you are sharing cant just tap the connectin it must acctulay connect to the hub puts more effort in to it.


Maybe this could be useful for non-public closed hubs. As i don't believe in closed hub, I didn't thought about this possibility.
Compressing files wouldn't be great benefit, because people tend to share files, that can't be compressed further (avi, mp3, rar, etc.). It would only add additional CPU load to both clients.
iso is the only significant file type which can be compressed. But compression ratio is usually very bad even for isos. And this would add significant complexity to both protocol and client implementation.
I did not meant this feat to be applayed as a check after the file transfer. The CRC shulde be sent to to downloding client befor it starts downloading and if it is a resume matched agains a stored CRC in the queue list to check that you dont resume from a bad copy.
Now I understand what you meant. This would be nice feature, but requires significant changes in protocol. I think there are still many areas that require less work to improve and bring greater benefit. I have heard some p2p programs use this and I don't know how exactly it is implemented. But I think that crc calculation and verifying whether file is changed adds unnecessary overhead.
I thnk the chat as it is today is good but what i ment was there is no need to extend it to use special chat channels and sutch in a IRC fashion. I have seen ides for this somewhere and i just wanted to point out that i dont think its a good ide.
I agree with you. If anyone wants better chat client than she should use irc. I think DC++ chat is quite usable now.
In the age of super-boredom/hype and mediocrity/celebrate relentlessness/menace to society --KMFDM

ivulfusbar
Posts: 506
Joined: 2003-01-03 07:33

Post by ivulfusbar » 2003-01-21 15:31

i don't have time to start a protocol-project right now, as i'm writing a book and that takes up all my time until its finished if it ever gets finished.

I have noted that there is no problem to write a new protocol thats better than the current protocol and have lots new server implimented by different people. The problem is that there are far more people interested in writing servers than clienta, and i don't blame them.

writing a server is alot more _easier_ when it comes to feedback, feature requests... i would never dare to start a project as arne has done,

sooner-or-later-someone-beginns-and-the-rest-will-follow-ly'ers ;))
Everyone is supposed to download from the hubs, - I don´t know why, but I never do anymore.

Skyfw
Posts: 1
Joined: 2003-02-27 19:51
Contact:

Post by Skyfw » 2003-02-27 20:55

You tend to write the RFC first, and then write the protocol to match, and not the other way around. This way you have a set of rules and goals to adhere to, instead of the patch and glue that is the current affair of the DC protocol.

As for DC Protocol V2, there seem to be plenty of people who are willing to, or already working on, a DC-related project. The main problem that i see here ( and of open source in general ^_^ ) is that everyone is working by themselves :P

So, a new protocol. Step 1. Write the RFC now.

Do a tech doc, write down what you are going to put in, and why. Write down how things should be done, and why. Submit doc to the net for critique. Chances are changes will be needed. Tech Doc V2 with changes generally agreed upon by the community.

Divide people willing to work into two general teams, one client, and one server. Have on-line space to place work so that both teams can view it. Make software, squish bugs, release beta, test, squish more bugs, release public.

volkris
Posts: 121
Joined: 2003-02-02 18:07
Contact:

Post by volkris » 2003-02-27 23:34

And don't forget that a good part of what you're trying to do has been done before. Someone else (probably someones else) has already written lengthy webpages on very similar topics to what you're proposing.

For the record I'm not horribly enthusiastic about a DC V2 project because I don't see the DC protocol as horribly flawed. The system it creates is, but not the protocol itself.

volkris
Posts: 121
Joined: 2003-02-02 18:07
Contact:

Post by volkris » 2003-02-27 23:43

Neg wrote:Public/Privte key eles they can log the key and the encrypion is totaly worthles.
Neg, the standard practice is to transfer the key using private/public keysets and then go with symmetric encryption from there on out. Private/public key systems are just too resource intensive. If the various communication endpoints have enough cycles to spare to do the encryption then sure, add it.
I did not meant this feat to be applayed as a check after the file transfer. The CRC shulde be sent to to downloding client befor it starts downloading and if it is a resume matched agains a stored CRC in the queue list to check that you dont resume from a bad copy.
Please, for gods sake, strike the word CRC from this discussion immediately. These days CRCs are for kiddies and others who don't know better, or for when cycles are incredibly precious. They aren't in any computer running DC++.

CRC is bad. MD5 and SHA are very, very good. Tiger tree is also very, very good (from what I know of it). Hashes would add a tremendous amount to DC, and including them in any future moves should be one of the top priorities.

Neg
Posts: 20
Joined: 2003-01-19 07:05

Post by Neg » 2003-02-28 10:47

volkris wrote:
Neg wrote:Public/Privte key eles they can log the key and the encrypion is totaly worthles.
Neg, the standard practice is to transfer the key using private/public keysets and then go with symmetric encryption from there on out. Private/public key systems are just too resource intensive. If the various communication endpoints have enough cycles to spare to do the encryption then sure, add it.
That is what i was aming for, i dont know mutch about cryptografi. But what i was trying to say was that the crypto prosses somewhere needed a public/private exchange so the key is not sent in 'plain text'.
volkris wrote:
Please, for gods sake, strike the word CRC from this discussion immediately. These days CRCs are for kiddies and others who don't know better, or for when cycles are incredibly precious. They aren't in any computer running DC++.

CRC is bad. MD5 and SHA are very, very good. Tiger tree is also very, very good (from what I know of it). Hashes would add a tremendous amount to DC, and including them in any future moves should be one of the top priorities.
Well i dont know mucth about how to check for file integrety and sutch things nither. And since we are not talking about implantation of any techonlogy only hypotetical layout of a new protocol i think CRC tells what i wanted to say. But if MD5 or SHA dose the work beter we shulde use it in a inplatatin of a new protocol. But when that come i hope experts in each area will step forward and talk about the actual implentation of the new protocol.

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Post by GargoyleMT » 2003-02-28 12:00

Neg wrote:
volkris wrote:... i dont know mutch about cryptografi.
...
Well i dont know mucth about how to check for file integrety and sutch things nither.
If you don't know what you're talking about, please just lurk and avoid posting unless you know that you're adding something of value.

GargoyleMT
DC++ Contributor
Posts: 3212
Joined: 2003-01-07 21:46
Location: .pa.us

Post by GargoyleMT » 2003-02-28 12:01

I should've used preview, as I neglected to cut/paste properly :oops: Those are Neg's quotes, they don't belong to volkris.

Neg
Posts: 20
Joined: 2003-01-19 07:05

Post by Neg » 2003-02-28 12:30

GargoyleMT wrote: If you don't know what you're talking about, please just lurk and avoid posting unless you know that you're adding something of value.
Well i talk about the hypotetical layout of a new protocol not the implantation of it. And i dont think it is significant to argue about what brand of a sulution is implanted in the final product when we talk about a teoretical layout.

The questions shulde be more like do we need this? why? why not? This is this type of questions i consider valueble at this stage. Not if we are going to use CRC or MD5 or whatever technology you can think of to do the task at hand.

And i think if everyone that was not an expert just lurked around the formus the number of valuble things that wulde get discussed woulde be mucth less then they are today.

sarf
Posts: 382
Joined: 2003-01-24 05:43
Location: Sweden
Contact:

Post by sarf » 2003-03-03 12:03

Neg wrote:Well I talk about the hypothetical layout of a new protocol not the [implementation] of it. And I [don't] think it is significant to argue about what brand of a solution is implanted in the final product when we talk about a theoretical layout.
To be quite frank, yes, you are right - it should not be significant - yet it is. Why? Well, its mainly because the thing people use is not the well-thought-out thing, it's the first thing they can get their hands on that does what they need done. So, while talking about CRCs and meaning "some way to ensure file integrity as well as providing a mean to uniquely identify files" is not really that bad, it's better to say "file integrity thingy and unique file identification stuff" when we're at this high level, otherwise people will read the text you are creating and say "CRC sux0rs bigtime! y00 all b3 moi bitch3s!" or just plain implement what you are talking about, and then where'd we be?
Neg wrote:The questions shulde be more like do we need this? Why? Why not? This is this type of questions I consider [important] at this stage. Not if we are going to use CRC or MD5 or whatever technology you can think of to do the task at hand.
Then let us leave the CRC/MD5/SHA things for the low-level bozos that will try and fail miserably in implementing our grand scheme.
Neg wrote:And I think if everyone that was not an expert just lurked around the formus the number of [important] things that [would] get discussed would be much less then they are today.
Perhaps you are right, but the posts about "HELP! MY DC HUB HAS BANNED ME! PLZ HELP!" would lessen dramatically, creating an environment in which thoughtful, poignant and interesting conversation could be done... while scaring away everyone that has not made one or two modified/hacked DC clients themselves.

Just remember - do not do too much high-level thinking or people will start using what's available and not the perfect thing you are trying to create.

Sarf
---
Never give an inch!

Locked