Re: [dcdev] New Encoding Scheme First
[email protected]
2003-10-27 10:23
Direct Connect developers

At 08:29 AM 10/27/2003 +0100, you wrote:
why not switch to a 100% text based client (with CR as command separator), someone mentions it earlier. And in any case, why not using XML based protocol, it is fairly easy to handle, well defined (without incorrect interpretation error), easy to extend and provides native UTF8 encoding.

that acts on or generates a message, but should have to edit the code that
encodes/decodes messages.

Requirements would be
     - EVERY hub that supports the new encoding scheme MUST support
       the old scheme
     - Every message is encoded/decoded with the same set of rules
     - No binary values

you have a problem here. You say you only want to modify the function that encodes/decodes message but this implies a generic function and a generic function will not convert the $MyInfo binary value into a non binary one.

     - One-to-one mapping from old messages to new messages
     - One protocol addition. A new message 'Supports' that identifies
       the commands a client supports sent immediately after it is decided
       that the new encoding method is being used.

I wholeheartedly agree with the idea of replacing DC's encoding with a more extensible sort.

However, some issues with the specific suggestions presented:

I see no reason to mandate that new hubs support both the new and old encodings, and certainly not indefinitely. DC client turnover is quick enough that I'd be comfortable in a few months with dropping support for the old encoding, for example, in any hubs I'm involved with.

CR should not be a separator; I find multiline message support valuable, for example, for ensuring pasted source code is nicely formatted.

XML, of course, avoids the problem with the proposed CR-delimited message format, and I find it preferable. Further, the suggestion to use lzo (or even zlib, depending on overhead; I know lzo has significantly less, but zlib's improved compression may be worth it. Something to consider), renders XML quite bandwidth-friendly. It has the additional benefit of encoding commonly used nicks and command and such concisely after it's built up a dictionary, beyond just the XML tags.

However, I'd prefer an a binary encoding to either of those, as a means of resolving the currently extant divide between a socket mode for transferring files and one for transferring commands. This may be an aesthetic point, but I find that quite unelegant. If one is going to homogenize the rest of the protocol encoding, leaving out the file transfers themselves strikes me, at least, as a conspicuous omission which should be justified. Mandating purely textual content, UTF-8 or not, would prevent such a unification of the protocol, and as such should be avoided. By contrast, a protocol designed to natively handle binary data would be quite capable of dealing with file transfer data itself in a common framework as the commands itself, and is thus an improvement over a text-basis for a P2P file transfer protocol.

My broader objection is the insistance on maintaining a one-to-one mapping between commands in the current encoding and commands in the newer encoding. Rather, I view this as an opportunity at least to remove useless commands (which are by my count broadcast $Hello, $GetInfo, $Key, $Lock, $Version, $GetListLen, $MultiConnectToMe, and $MultiSearch at minimum. I'm sure many of you disagree with various listed items, but there has to be a subset nearly all on the list would be willing to remove).

Finally, DC has many semantic, rather than syntactic, limitations, and whilst replacing the syntax certainly aids towards fixing the broken semantics by creating a more extensible environment in which to introduce improved semantics, I feel that one should take the opportunity created by breaking compatibility to introduce functionality such as: standard GetZBlock-type download support, hash support, search ID support, and various other flaws described at http://murdoc.extatic.org/dctng.html.


DC Developers mailinglist