Efficient Test-Time Adaptation of Vision-Language Models