Every now and then it has been suggested that one of the VM’s enciphering steps included some kind of anagramming, and it has been surmised that “already at Galileo’s times, this method was used to hide information”. The latter is not quite true.
(Much of what follows is lifted from Wikipedia.)
What researchers in the early modern age did was to establish priority of discovery in their anagrams. For example, Galileo noted that Saturn looke quite oddly in his newly-assembled telescope, and hence rushed to publish a note saying smaismrmilmepoetaleumibunenugttauiras some place before he went on to more intense, detailed and time-consuming studies. Nobody could decipher what this was supposed to mean (Heed this, we’ll talk about this again later on!). The plaintext was Altissimum planetam tergeminum observavi (“I have observed the most distant planet to have a triple form”, with Saturn being the outermost known planet of the solar system at the time, and the appearance of Saturn’s rings being that of two smaller satellites on either side of the planet in Galileo’s small telescope.)
By this method he could safely go on with this research. He had avoided putting other astronomers on the track of what he was examining, while he could at the same time still claim priority of the discovery by explaining the anagram — should somebody else in the meantime happen to make the same discovery by accident.
While this apparently worked nicely and was an established mode of operation back then, it also shows how useless random anagramming is for the encipherment of a document: To decipher it is as difficult for the legitimate receiver of the message as for any wannabe code-breakers. (Or, if you want to, the ambiguity introduced is as large for both of them.)
Note that this holds only for “lossy anagramming”, where there is no rule to retrieve the original letter arrangement again. (As for example when sorting all letters of each word alphabetically.) Non-lossy anagrams, which involve rearranging the letters according to reversible rules (For example, yb wstihcnig aehc apri fo elttres in a word), of course allow for a perfect retrieval, but these tend to erode any underlying plaintext word structure, rather than enhancing it — the latter is observed in the VM.
Thus, I’m pretty convinced that anagramming doesn’t play a major role in the VM’s enciphering scheme. My best bet is still some fiddling with the character set which leads to combinations of ciphertext characters representing a single plaintext character.
(Did anybody else note that this “Priority by anagramming” seems to work a little like PGP-encrypted messages?)