Reliability and validity of your thesis

Describing the reliability and validity of your research is an important part of your thesis.

Students in higher professional education as well as for academic students are required to describe these, and both are usually discussed in your methodology.

We help students with this daily, because describing these research concepts is often not too hard, but using them? That’s a different story!

In this article we explain these concepts and give you tips on how to use them in your thesis. Good luck!

Reliability, an example

When you look up the term reliability in you research manual, you will often find different definitions.

In the end, what matters with reliability is that the results of your research match the actual situation as much as possible. When a fellow student would research your topic, you want him or her to obtain practically the same results. Only then can we speak of a reliable study and will your research be reproducible.

For example: You are studying the dietary habits of vegetarians in the Netherlands. You decide to conduct a survey and you use social media (Facebook) for this purpose. You notice that it’s mainly your fellow students, from your college town, that reply to your call to participate. But is that reliable? You want the results to mirror the actual situation, which concerns all vegetarians in the Netherlands. Because it’s mainly students that have participated in the study, its reliability is low. To obtain results that mirror the actual situation as closely as possible, it would be better to add other respondents and for that you could use other methods of selection, like requests on internet fora, additional paper surveys etc.

Reliability in quantitative research

Reliability in quantitative research is often expressed as a reliability percentage of 95% or 99%. Do you want to know how many respondents you need to achieve this reliability score? You can use a sample size calculator, for example using this link.

You can also test the reliability of questionnaires in SPSS by calculating the so-called Cronbach’s Alpha. If this value is .80 or higher, this indicates a high reliability.

Most literature indicates that a score of 0.70 or higher is also still reliable.

Watch out: you can’t just combine all questions (with different response categories)!

Do you want to get the reliability (and validity) of your research on paper, together with a thesis-expert?

Contact us now…

Validity in quantitative research

The validity of quantitative research usually has to do with which questions you ask your respondents. Validity means that the results you find will be factually correct.

It is often best to draw up your questions after you have finished your literature review (which is also most reliable!). You can then better judge which concepts are important, and use these concepts to draw up different questions (indicators). While doing this, always keep in mind the goal of the study and your sub-questions.

We have previously discussed the methodological validity, but you can also – this applies especially to academic students – look at the statistical validity.  For this you check whether the used statistical models meet the underlying assumptions.

Reliability and validity in qualitative research

Reliability is just as important in qualitative research as it is in quantitative research. In discussing this, you need to describe certain things regarding the circumstances of your research, like for example the fact that a quiet location was chosen (so that respondents could not be disturbed), the attitude of the interviewer (respondents should not be influenced, so take on an open attitude, ask open question instead of guiding ones, stay silent in between, etc.). It is also important that you describe the representativeness of your sample, as the representativeness of the sample makes for better reliability. If you wish, for example, to map the wishes and needs of a company’s target group, try not just to include current customers in your study but also and especially potential new costumers.

Furthermore, it is good to discuss why you have specifically chosen this research method and what the pros and cons of this method are.

When using interviews, for example, you can choose between structured, semi-structured or in-depth interviews. The goal is to justify your choice as well as possible. (Tip: it often helps to describe the purpose of the interviews.)

As with quantitative research, validity in qualitative research is about drawing up the right questions/topics that really cover your subject matter. Here too you can make use of an operationalization model. The interview questions (with structured and semi-structured interviews) are often listed in an interview-guide, to help you go to an interview well prepared. With in-depth interviews it is common to use an item-list.

In this document you can also write down for yourself an introduction that you give each interviewee at the start of an interview, and you also mention the confidentiality. See the image below for an example of an interview-guide.

Finally you need to describe here how you are going to perform the analysis and whether you have for example recorded the interviews (and transcribed them).

Moreover, it is always good to present your interview-questions to an expert and to do some trial interviews if possible.

Describing reliability and validity in your thesis

To summarize, discuss as many aspects as possible that have led to the highest possible reliability of your study. Include at least the following:

Why you have chosen this type of study (qualitative/quantitative), more information on the type of research (so not just “interviews”, but for example “semi-structured interviews”) and the reasons for these choices.

The reason for the number of respondents (for quantitative research you can use a sample size calculator, for qualitative research you can theoretically explain this.

Whether you have done a pre-test/trial (survey/interview)

How you have selected your respondents (social media, notices, etc.) and how you have dealt with the privacy of your respondents (anonymous or not).

A justification of how you have drawn up the questions (of your survey/interview/etc.). Click here for tips on how to draw up good survey-questions.

The representativeness of the respondents in your study (sample).

The response percentage: How many people you have invited to participate (if you can check this, for example when you have sent out a request via e-mail) and how many have actually participated.

What you have done to heighten the response rate: to get more responses (and to make sure your sample becomes more representative), it often pays to raffle off a gift voucher among the participants and to send a reminder (for example one week after sending out the invitation). You will see that this will increase the number of people that will participate in your study.

How you have analyzed the results (for example using SPSS in the case of quantitative research, or in the case of qualitative research based on interviews you can describe how you have coded the replies etc.).

Reliability of literature review (secondary research)

The definition of a literature review according to Topscriptie is: An overview of already existing information on your subject (what is known already) that shows the context of and identifies ‘gaps’ in the literature, takes the form of a critical and coherent review and sheds light on the connections between previous studies.

Many students forget to describe how they have tried to keep the reliability of their literature review (desk research) as high as possible. It is important to discuss this in your thesis. Describe for example the following aspects:

Nowadays you can find a lot of incorrect information on the internet, Google cannot always check the source of something and everyone is allowed to make a website and publish something. It is therefore good to discuss that you have used objective sources as much as possible, for which the author has no or very little commercial interest and that you have specifically paid attention to this when selecting your literature.

 If you have used (a) database(s), indicate which one(s) you have used and why. Additionally, you might have used Google Scholar instead of Google.

 Indicate which keywords and combinations of keywords you have used. (Pay attention to the use of AND and OR!) It is also often worth it to search for your keywords in English as well as in Dutch.

With the above mentioned tips from Topscriptie you can get started yourself on describing the reliability and validity of your study in your thesis. Don’t forget to discuss the limits of your study in your discussion. It actually makes your article more solid when you can show that you are aware of these. Don’t forget that every thesis and every study has its own problems with the reliability and validity.