I wonder if the RGMII clock master is different on RX vs. TX. This could explain it, or my hunch could be completely wrong and the issue caused by something very different...
-
-
Show this thread
-
Twitter is the best rubber duck. After multiple nights of debugging, I find the problem 15min after posting it here...pic.twitter.com/tWFN7oGD5F
Show this thread -
The "ID" stands for "Internal Delay". RGMII PHYs are usually either with or without internal delay, and the MAC needs to match these settings. Looks like my MAC is expecting delay on RX and TX side.
Show this thread -
The PHY datasheet has useful diagrams about what the internal delay does. Not surprising that my RX data was screwed:pic.twitter.com/8ehMLY6gFr
Show this thread -
Now, why was the Linux driver for my NIC not hitting this issue on 4.2.8? That's because the PHY driver was changed in Linux 4.5. Before, it always enabled RX delays (default PHY setting on reset). After, it explicitly disables RX delays if not requested.pic.twitter.com/DzrAHxpfGP
Show this thread -
"Fun" fact, I looked up the mailing list thread for that patch, and it broke at least one more board in exactly the same way, starting a discussion on how that change could have been better introduced. https://www.spinics.net/lists/arm-kernel/msg594740.html … Wish I'd have found out earlier...pic.twitter.com/7T9eCi5hja
Show this thread -
Anyway, now my PHY and MAC can talk to each other. Just needs to fix my MAC and the Linux network stack not understanding each other :-) There is probably some SKB manipulation snafu left: the packet data seems to end up getting duplicated within the same packet.pic.twitter.com/pbmRqKMAF2
Show this thread -
Removed what I thought was dead code to simplify the driver code (there are multiple SKB receive paths in the "upstream" code to support old Linux versions that don't have e.g. GRO), problem fixed. That was easy. I hate not understanding the fix, but whatever.
Show this thread -
It's probably the 5th time I mention this feature, but NixOS is just amazing sometimes: $ nix-build '<nixpkgs>' -A pkgsCross.aarch64-multiplatform.pkgsStatic.iperf $ file result/bin/iperf3 ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
Show this thread -
[SUM] 0.00-10.00 sec 574 MBytes 482 Mbits/sec receiver [ 5] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec sender RX performance isn't great, but... good enough for now.
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.