Flapping in orthographic Verb+to constructions reflects a lexicalized cliticization process, as exemplified by the famous example "Teddy is the man I wanna win" -- ungrammatical except if Teddy is the object of "win". You can do that with woulda, should, gotta, hafta, oughta, coulda, wanna, and these do not always correspond to "to". If the verb is vocoid-final (with no intervening trace), you can flap in casual speech as in "try to go", "not far to go", "say to the boss" where flaps are possible. But you won't find General American English speakers doing this with d- and t- final verbs apart from cliticization case, so not in "forget to close the door", "compute to 5 decimal places", "set to music".