The Web is becoming the preferred delivery mode of learning management services and information distribution portals. Intensive research has been done towards improving the usability of these Web sites. Post-test questionnaires are often used as a tool in evaluating e-services. In this paper we investigated the reliability of post-test questionnaires as an evaluation tool in the South African context. The study was done at a South African, multi-cultural open-distance learning university. We conducted two usability studies that included formal laboratory testing, eye tracking, and post-test questionnaires.  The first study investigated the usability of an assignment submission learning management system while the second study focused on the usability of the university information portal. Our participants were a multi-cultural group that represented 11 different languages. Ten out of the 11 were official South-African languages. Data capturing included live video recordings, audio in the form of the participant or the test administrator speaking, eye tracking video recordings, eye tracking data files and a post-test questionnaire.  The data was analysed to evaluate the usability of these two websites. Triangulating the findings from the formal laboratory testing, eye tracking, and post-test questionnaires within each study we concluded that the findings based on the post-test questionnaire data was not consistent with the findings based on the formal laboratory testing and the eye tracking.  According to the post-test questionnaire data in both studies, the participants reported a positive experience with the use of the websites. However, the formal laboratory testing and eye tracking data showed that several participants had difficulty in using the websites.  These difficulties included problems such as navigation, finding information and understanding the terminology. The problems were so severe that they needed assistance in order to continue. This discrepancy raises questions about the reliability of post-test questionnaires.  The contribution of this paper is to highlight the problems with using post-test questionnaires in a South African context where the results can be heavily influenced by social desirability bias and acquiescence.  This has implications for e-learning, e-government and other websites that provide services to a diverse population.