Many factors can play a role in this. A root bound tree with throw new growth then dwindle rapidly. It may be time to repot if you haven't done so in a while with a fresh potting substrate. Another cause for leaf drop is cold temperatures, inconsistent watering or over watering. Even a rapid change in growing conditions can stress a tree enough to drop some leaves.
Do not be concerned about the flowers falling off. A lemon tree will blossom a million times a season and out of those blooms, only a few will be successful and produce fruit. Like any other fruit tree. The amount of fruit set on the tree is determined by what the tree can sustain via it's root system. The bigger/healthier the roots the bigger the fruit set.
Also watch for pests! Ants and spider mites love themselves a nice lush fruit tree to feast/live on.
Hope I can help