Found an old (1989) paper about "must of". Interestingly, the author says he had trouble finding examples in print, but those were pre-internet days! (Here it is in Steinbeck, and here's a virtual tome called "Must of Got Lost".)
Coates suggests that since "of" is a preposition, speakers who use it in this construction have to somehow make sense of a preposition being in this position, but I doubt it. Language is full of idiom: phrases which don't make much sense if broken down into their component parts, and it could be argued not only that "must of" is simply accepted by its users as having that function, but also that "must have" is itself by way of being an idiom.