A downfall is something that causes a person’s destruction, either literal or figurative: “expensive cars were Fred’s downfall: he spent his entire inheritance on them and went bankrupt.” A drawback is not nearly so drastic, just a flaw or problem of some kind, and is normally applied to plans and activities, not to people: “Gloria’s plan to camp on Mosquito Island had just one drawback: she had forgotten to bring her insect repellent.” Also, “downfall” should not be used when the more moderate “decline” is meant; reserve it for ruin, not to designate simple deterioration.