After years of working on learning Portuguese, I still have trouble in using the correct past tense. I came across something in Wikipedia that explains why English speakers as a first language (like myself) may have problems:
Romance languages
«A common mistake of beginners learning a Romance language is putting too much emphasis on whether the time the action occurred is known. This generally does not affect how the imperfect is used. For example, the sentence "Someone ate all of my cookies. " (when translated) is not a good candidate for the imperfect. Fundamentally, it is no different from the sentence "We ate all the cookies. " Note this fails the repeatability requirement of the imperfect, as it is only known to have happened once. On the other hand, the sentence "I used to have fun in the 1960s. " is a good candidate for the imperfect, even though its period is known. In short, knowing when an action occurred is not nearly as important as how long it occurred (or was and still is occurring).»
Full WIkipedia article: https://en.wikipedia.org/wiki/Imperfect