Hold on - this isn't right..... Why is Google Authenticator my problem?
In order to get to the /administrator login form, you MUST have already entered the URL key(s) or added yourself to the whitelist. Once AdminExile has cleared you to view the /administrator login form, it ignores you. Whatever you're putting into the login form that's letting you in after 2 attempts is a function of your authentication plugins - not AdminExile. AdminExile isn't an authentication plugin, it's a system plugin. It doesn't have any facility to log you in - it only prevents or allows the display of the /administrator login page.
If the Google Authenticator plugin is malfunctioning, you should contact the author of that plugin.