UdpMcast
From Wiki for RobotCub and Friends
YARP supports UDP and MCAST communication. These are unreliable network protocols, but can be quite efficient. We discuss here steps that are useful for making your network friendly to UDP and MCAST. But if you find yourself spending a lot of time on this, you may need to reconsider whether you have made the right choice of protocol.
Contents |
Check List
- Do all your computers permit programs to allocate large receiver buffers? 640x480 color images at 30Hz just won't work in UDP/MCAST without a big receiver buffer, if your program ever does any actual processing. See "YARP's suggestion" below.
- Do you know your Maximum Transmission Unit (MTU) Size? See "Useful Links" below.
- Have you become familiar with the output of the network throughput test program iperf on your system? Again, see "Useful Links" below.
Using iperf
Checking TCP
Example of use: pick two machines, choose one (arbitrarily) to be the server. Find its IP address or domain name -- let's say it is 10.0.0.26. Run the following commands:
[On server, let's suppose it is 10.0.0.26] iperf -s [On client, let's suppose it is 10.0.0.27] iperf -c 10.0.0.26
This will run for a while and then describe your TCP throughput, for example:
------------------------------------------------------------ Client connecting to 10.0.0.26, TCP port 5001 TCP window size: 16.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.25 port 34160 connected with 10.0.0.26 port 5001 [ 3] 0.0-10.0 sec 1.08 GBytes 924 Mbits/sec
Checking UDP
Now let's try UDP. Add the "-u" flag to the client and server. To the server, we can add "-i 1" to get reports at regular intervals. To the client, you can specify target bandwidth, e.g. "-b 50m" for a target of 50 MBits/sec.
[On server, 10.0.0.26] iperf -s -u -i 1 [On client, 10.0.0.27] iperf -c 10.0.0.26 -b 50m
Example result:
Sending 1470 byte datagrams UDP buffer size: 108 KByte (default) ------------------------------------------------------------ [ 3] local 10.0.0.25 port 32820 connected with 10.0.0.26 port 5001 [ 3] 0.0-10.0 sec 59.7 MBytes 50.0 Mbits/sec [ 3] Sent 42555 datagrams [ 3] Server Report: [ 3] 0.0-10.0 sec 119 MBytes 100 Mbits/sec 0.030 ms 0/42554 (0%) [ 3] 0.0-10.0 sec 1 datagrams received out-of-order
A key test for YARP is that you can send big datagrams okay. This is necessary for efficient transmission of large images using UDP or MCAST. To check 63KB datagrams, do:
[On server, 10.0.0.26] iperf -s -u -l 63K -w 200K [On client, 10.0.0.27] iperf -c 10.0.0.11 -u -l 63K -w 200K
Make sure you try this in both directions! We've seen problems going from a Linux machine to a Windows machine (upgrading kernel fixed it; maybe NIC driver issue?). If you see problems, reduce the "-l" size until things work just to understand your system, but if you want to send large images you'll need to tweak your configuration.
Checking Multicast
For testing multicast, make sure you have a version of iperf >= 2.0.2. Try this:
iperf -c 224.0.55.55 -u -T 32 -t 100 -i 1 -l 60k iperf -s -u -B 224.0.55.55 -i 1 -T 32 -l 60k
On the client ("-c") side, you should see something like:
------------------------------------------------------------ Server listening on UDP port 5001 Binding to local address 224.0.55.55 Joining multicast group 224.0.55.55 Receiving 61440 byte datagrams UDP buffer size: 108 KByte (default) ------------------------------------------------------------ [ 3] local 224.0.55.55 port 5001 connected with 10.0.0.25 port 32821 [ 3] 0.0- 1.0 sec 120 KBytes 983 Kbits/sec 0.003 ms 0/ 2 (0%) [ 3] 1.0- 2.0 sec 120 KBytes 983 Kbits/sec 0.030 ms 0/ 2 (0%) [ 3] 2.0- 3.0 sec 120 KBytes 983 Kbits/sec 0.052 ms 0/ 2 (0%)
YARP's suggestion
If MCAST/UDP packets drop, YARP may suggest this:
The UDP/MCAST system buffer limit on your system is low. You may get packet loss under heavy conditions. To change the buffer limit on linux: sysctl -w net.core.rmem_max=8388608 (Might be something like: sudo /sbin/sysctl -w net.core.rmem_max=8388608) To change the limit use: systcl for Linux/FreeBSD, ndd for Solaris, no for AIX
This controls the maximum allowed size for a RECEIVER buffer, so that a number of packets adding up to a large amount of data can be received and stored while your program is doing something else. YARP will try to get a large buffer, and your operating system may not comply unless you do the above. This configuration step is not necessary on MS-Windows.
Testing bandwith for UDP and Multicast
The tests above have concentrated just on connectivity. To test whether a specific data rate is possible, add the "-b" option. This applies to UDP and multicast. For example, on a gigabit network, to see how things work with 500MB/s of data, try adding:
-b 500m
Relevant source code
In case you need to fix something, the relevant source code in YARP is:
- src/libYARP_OS/src/DgramTwoWayStream.cpp
- src/libYARP_OS/include/yarp/os/DgramTwoWayStream.h
We appreciate patches to these files. We only have one network to test on, and this stuff seems quite network-dependent.
Useful Links
- iperf is a useful network throughput testing tool
- The Maximum Transmission Unit (MTU) Size on MS-Windows
- UDP/TCP Network Tuning on Linux
