that which annoys me

Carol Ann Wilson is 30% right: where there's a comma, only "which" is possible. However, where there's no comma, both "that" and "which" are correct: "The report which I sent last week will be of some help." is a fine sentence. Ignore any guide which tells you otherwise.