Do you have to be physically fit to start?

No. You only train at a pace that best suits your own level of fitness. The Universal Self-Defence Academy also teaches you to focus more on improving your own ability rather than competing against other members of the class.
This is one of the reasons why Usda training is more enjoyable, safer and easier to learn and improve. If you have any health problems or any concerns about physical limitations or fitness, then you should consult with your physician before you join Usda or participate in any physical activity.