What ARE the differences really? They all seem the same to me basically. Although until recently when I finally hit 566 kg on large (and could never get close again!), it seemed easier to get bigger in medium than large.

the main differences are:

where you start,
in small there are no giant dango thingies in the farm part,
in medium there are melons and cabbage in the farm area,
in large there are giant peaches and parfaits in the farm area,

i don't think there is difference in time and yes it is much easier to size up in medium because there are giant dangos and small pickuppable foods in the farm area. these small things really affect your ending size because it determines the amount of time you waste getting big enough to pick up the food stands.