I haven't done this route. It's easier from Tokyo to go to Fujisan or Kawaguchiko stations and start from there, either the Subaru, Yoshida or Azami routes.
But judging from Strava Heatmaps, people who cycle the Fujinomiya route do so like this
The first 10 km is just plain ol' grey Japanese city. After that it will get progressively nicer. It will get cold, too. I see from your avatar pic that you are no stranger to Fujisan in November. But the weather can change very quickly. Please prepare well and don't hesitate to call it off if things look like they might turn nasty.