We stopped in Kumagaya to refuel, and then took Rte.341, heading almost directly north for the next 10km to Tonegawa. We crossed Tonegawa at 10:15, and turned left onto the cycling road there. This was probably the most difficult part to navigate, as there are several tributaries that feed into the Tonegawa river itself. This section of Tonegawa basically runs west to east. We wanted to be heading directly west, but the cycling road tends to follow these tributaries, branches of the river, north. Several times, we had to cross over, and then head back down the other side, back to the Tonegawa river proper. The cycling road ends after about 30km, and we made our way into Takasaki along Rte.121 (a.k.a. The Togoku Culture & History Hwy).
Upstream to the (very unceremonious) end of the Arakawa cycle road at Kumagaya, up the 341 to Tonegawa (a bit farther than 10km, actually), and then left upstream the Tonegawa? That's my route. But there's a difference: I don't cross the Tone; I instead go up its right bank. Bandō-ōhashi (坂東大橋, for route 462) is upstream from you and you'll never be in any risk of forgetting this: again and again you're told how far you are from it. At Bandō-ōhashi, cross the river, and continue upstream along the left bank. The first bridge you get to is Goryōbashi (五料橋, for route 354); you can't miss this either. At Goryōbashi, cross back over, and continue upstream along the right bank. This will take you all the way to Takasaki (and beyond, if you wanted.)
Whether on the one bank or the other, the route is pretty much free of annoyances from where you get on to it (Tōsuibashi, 刀水橋) all the way upstream to Goryōbashi (if I remember right), but it then runs alongside houses and there are more unexpected obstructions, people walking their dogs, etc, all the way to Takasaki. But it's nothing compared with a Sunday along the lower reaches of the Tama. Take it slow. The only real oddity is that somewhere within Takasaki-shi there's a very minor (and well-signposted) diversion away from the river, and you turn sharp left to be suddenly faced with an unexpected slope. The first time I encountered this, I was so zombified by perhaps a hundred km of uninterruptedly level cycling (aside from a bridge over railway tracks in Kumagaya) that I couldn't change gear or even unclip myself in time, and fell over. (Duh.) However, there are no U-turns caused by tributaries, and there's no difficulty whatever in navigating.
(Here's a PDF of the official cycle route map, actually published by Gunma prefecture. It's too small to be very useful, but you can see that all I'm doing is unimaginatively following their route.)
If, with a different destination, you wanted to keep going upstream, you'd find that from Maebashi the route speeds up again.