The expression is “hardly ever” or “almost never.”