ADC inform, an idea
Moderator: Moderators
-
- Posts: 52
- Joined: 2003-03-12 00:06
- Location: Zinzinnati
ADC inform, an idea
I had an idea about easing the hub load on new connections with ADC. Right now the hub sends the entire user list with all their information to the newly connecting client, right. A way to cut down the hub traffic is to have the hub instead inform the existing clients about the newly connected client, and have them tell the new client about themselves. That way the user info list is a client<->client communication rather than a hub<->client. So for example client A connects to the hub, with all its information, and a 16 bit random string to identify the login. The hub then sends a message to the existing clients, an "inform request", containing the new client's IP, port, and the 16 bit string that A originally sent, telling them to send all their information to client A. Client A will also give out all its information in this exchange. Client A knows which hub the replies go with based on the 16 bit string. This way the hub has the full client list and all the clients have the full list, but with a minimum of hub traffic. For passive clients, a reverse form of this can be used. For passive<->passive clients, the hub itself will have to give out the information.
Just an idea.
LA
Just an idea.
LA
Anata ga baka da! Repent!
-
- Forum Moderator
- Posts: 1420
- Joined: 2003-04-22 14:37
-
- Posts: 52
- Joined: 2003-03-12 00:06
- Location: Zinzinnati
Ur right, the total traffic is the same but it has been rearranged for the benefit of the hub. In the current way, all the user list information flows over the hub's inet pipe to the new client. This way, only the inform request goes over the hub's inet pipe and the full load of the user list is distributed to all the users. If this were a relationship diagram it would be like:
old: 1:1
new: 1:n:1
Ur second paraphrase is wrong though, or atleast truncated. To carry on ur terminology it would be "Give information about 1 user to n users; n users give information about 1 user (self) to 1 user."
To go into a bytes perspective, the hub now transmits all user information, which can be of significant length. nick+share+connection+tag+email+mode etc. But if the hub is "I'm not telling this guy about u, u tell this guy about u" then it only has to send the length of ip+port+key. e.g. XXXXX123.123.123.123411Gp. Or whatever the ADC consistent formatting is. But u see that the bandwidth savings for the hub can be significant. Even greater if packed forms are used.
LA
old: 1:1
new: 1:n:1
Ur second paraphrase is wrong though, or atleast truncated. To carry on ur terminology it would be "Give information about 1 user to n users; n users give information about 1 user (self) to 1 user."
To go into a bytes perspective, the hub now transmits all user information, which can be of significant length. nick+share+connection+tag+email+mode etc. But if the hub is "I'm not telling this guy about u, u tell this guy about u" then it only has to send the length of ip+port+key. e.g. XXXXX123.123.123.123411Gp. Or whatever the ADC consistent formatting is. But u see that the bandwidth savings for the hub can be significant. Even greater if packed forms are used.
LA
Anata ga baka da! Repent!
That's certainly a good idea in general, but there are several issues in practice. On a large hub, a newly connecting user would suddenly get thousands of incoming connections at once from clients trying to get their info through. Firewalls won't like that and socket systems tend to break or not work correctly under such stress. The resulting raw traffic (including overhead for establishing the connection) would be much higher. If you imagine a hub reboot situation with thousands of clients connecting in a short period, the problem multiplies.
Reliability of connections is also a big problem. Not everybody can establish connections to everyone, even if they're not both passive. There would be no way for a client to notice that the user list is incomplete because some clients couldn't connect...
...or simply didn't. There's also the trust issue ullner hinted at. The hub doesn't see the information and thus can't verify it.
A more practical solution would be to use clients as "sub-hubs" to divide the broadcast responsibility, but that's not trivial either.
Reliability of connections is also a big problem. Not everybody can establish connections to everyone, even if they're not both passive. There would be no way for a client to notice that the user list is incomplete because some clients couldn't connect...
...or simply didn't. There's also the trust issue ullner hinted at. The hub doesn't see the information and thus can't verify it.
A more practical solution would be to use clients as "sub-hubs" to divide the broadcast responsibility, but that's not trivial either.
-
- DC++ Contributor
- Posts: 3212
- Joined: 2003-01-07 21:46
- Location: .pa.us
Re: ADC inform, an idea
It has its own problem: how can you trust such information? How does the hub know you're sending out the same information to users as you send to it? (Not a whole bunch of bad can come of this - the stuff in INF is relatively benign, except for what Ullner mentioned.lordadmira wrote:I had an idea about easing the hub load on new connections with ADC.
Based on the existence of "bad" clients and DC++ derivations to detect them (DCDM), I don't think this is a step in the right direction.
-
- Posts: 52
- Joined: 2003-03-12 00:06
- Location: Zinzinnati
Not necessarily. The hub can throttle the rate as necessary. The connect-transfer-disconnect only takes a fraction of a second anyway.FarCry wrote:a newly connecting user would suddenly get thousands of incoming connections at once from clients trying to get their info through. If you imagine a hub reboot situation with thousands of clients connecting in a short period, the problem multiplies.
If two people can't connect to each other and they're non passive, then they're doomed from that point on. If I can't download from person X they're dead to me. I don't even need to know he exists, it doesn't do me any good.Not everybody can establish connections to everyone, even if they're not both passive. There would be no way for a client to notice that the user list is incomplete because some clients couldn't connect...
We have that problem now. If there is a malicious client they can already ignore search requests or downloads. There's no way to control what a client does....or simply didn't. There's also the trust issue ullner hinted at. The hub doesn't see the information and thus can't verify it.
LA
Anata ga baka da! Repent!
-
- Posts: 52
- Joined: 2003-03-12 00:06
- Location: Zinzinnati
-
- DC++ Contributor
- Posts: 3212
- Joined: 2003-01-07 21:46
- Location: .pa.us
We have malicious clients now, and they do send out spurious information in MyINFO, but I think you can appreciate that the solution you've proposed adds another level to their ability to mislead people - it would be quite easy to broadcast one thing to operators and another to users. Sure, that can be detected eventually, but I think that makes the solution too unclean.lordadmira wrote:We have that problem now. If there is a malicious client they can already ignore search requests or downloads. There's no way to control what a client does.
It might be wise to go back and look at the evolution of SCH, for although it was also proposed to be client to client, it's now advised to be used only in trusted environments. At least one of the concerns, flooding, is the same.
-
- Posts: 52
- Joined: 2003-03-12 00:06
- Location: Zinzinnati
"Informative" means anything not central to the operating of the hub system. It's just things "nice to know". U don't need to know any of MyInfo or even have the complete userlist to send search requests. IP and port is all u need for client<->client searches. Having a totally secure unspoofable network is against the laws of physics.
I guess I'm not averse to complexity for the sake of even moderate improvements.
I guess I'm not averse to complexity for the sake of even moderate improvements.
Anata ga baka da! Repent!
-
- Posts: 506
- Joined: 2003-01-03 07:33
Establishing a connection can easily take much longer than a second. The hub also doesn't know how many connections per second a user's setup (firewall) will tolerate. You surely don't want the user list of a 10000 user hub to take an hour to build, just to be on the safe side.lordadmira wrote:Not necessarily. The hub can throttle the rate as necessary. The connect-transfer-disconnect only takes a fraction of a second anyway.
He'll still get your main chat messages and you might even get search results from him. A connection may also work if it's established the other way around. The problem may also just be temporary and retrying for an unspecified time isn't a good solution. An operator or bot (I for one even as a normal user) would always want to see all online users, no matter if it's possible to get a connection to them.lordadmira wrote:If two people can't connect to each other and they're non passive, then they're doomed from that point on. If I can't download from person X they're dead to me. I don't even need to know he exists, it doesn't do me any good.
-
- Posts: 4
- Joined: 2005-12-05 10:08