When preceding a pronoun, all must be followed by of (all of them, not all them or all us). When all is followed by a noun, of may be omitted and often is by speakers and writers who wish to avoid using an unnecessary word: "All (or all of) the workers went on strike."
Dictionary of problem words and expressions. Harry Shaw. 1975.