More information would be helpful: what program are you installing? Generally speaking, when you use AutoIt to interact with a dialog box then the box will need to be visible. That's the nature of the beast. You can also use AutoIt to install a program silently if the program has appropriate switches: myprogram.msi /qn, for example. I use AutoIt to install everything, saves me the trouble of typing.
Edit: check MHz's sound advice
HERE. Looks like you can indeed make windows transparent using AutoIt, in effect creating a silent install. Sorry for my earlier "shoot from the hip" comments.