Conducting a Mixed Methods Study to Explore Staff Perceptions in Providing Care to Individuals With Co-Occurring Mental Health Problems and Illicit Substance Use

This case study examines key issues in designing, developing, and conducting a research study to explore the perceptions of inpatient staff to individuals with co-occurring mental health problems and illicit substance use. The use of mixed methods research is outlined and its application to this research area is reflected upon. The choices the researcher made are identified as well as what went well, the challenges along the way, and how the researcher addressed problem areas. Key learning points are summarized and in-hindsight observations discussed to support the student researcher.

1. Understand the strengths of using mixed methods research 2. Define "triangulation" in the context of mixed methods research 3. Analyze why the combination of quantitative and qualitative methods can facilitate a more comprehensive investigation Case Study

Project Overview and Context
This research study aimed to explore multi-disciplinary mental health staff experience and perceptions in caring for individuals who have mental health problems and who use illicit substances. I was interested in this area because in practice this was a current topic which was always under discussion in multi-disciplinary meetings and I was involved in these discussions as a clinician and manager. There were also attitudinal aspects which caused disharmony in formulating team therapeutic patient approaches. When I conducted a scoping literature review, I found some research relating to this area but it was not extensive, which supported the indication that further research was needed in this area. The methods chosen included a (quantitative) selfadministered questionnaire survey and (qualitative) semi-structured in-depth interviews. The rationale for this was to use the data received from these two contrasting forms of enquiry to provide an in-depth exploration. It was aimed that the questionnaire would ask specific questions on staff perceptions and incorporate questions on staff attitudes which would yield quantitative data, and the interview data would result in rich personal accounts from participants. The data obtained would be used to validate outcomes from each chosen method and to also investigate any discrepancies highlighted. This is often a primary reason for the choice of a mixed methods design in that it uses multiple approaches to answer a research question (Burke Johnson & Onwuegbuzie, 2004).
Section Summary  The research area was chosen as a result of practice experiences of the researcher and from initial literature review results which indicated a lack of research in inpatient staff perceptions of individuals with co-occurring mental health problems and illicit substance use.
 Questionnaire data (quantitative) and interview data (qualitative) enabled these two contrasting forms of data to provide an in-depth exploration of the perceptions and attitudes of the professional staff involved in the study.

Research Design
When designing the questionnaire component of the study, the first part of the questionnaire was designed to focus upon individual staff members' experiences and multidisciplinary working and problematic issues. For the second part of the questionnaire, I incorporated a standardized measure to look at perceptions of drug use. This involved the use of Watson et al.'s (2003) Drug and Drug Problems Questionnaire (validated version). This step was taken to add to the reliability and validity of the study. Reliability in quantitative research refers to the accuracy of an instrument and the extent to which a research instrument can consistently yield the same results on repeated applications within the same situations (Heale & Twycross, 2015). Validity pertains to whether the research is accurately measuring what it is intending to measure, that is, if the measure is supposed to be focusing on depression but is in fact measuring anxiety, it would not be viewed as valid (Heale & Twycross, 2015). By using a validated attitudes Likert-type scale, it was hoped that the results of my own study could be compared to previous studies which had used this same measure. It was also possible that comparisons could be drawn of subgroups of participants and that over time a follow-up study could be conducted following a specific training/education initiative. The methods were also designed and implemented so that the data obtained from the questionnaire could indicate subject areas of questioning for the qualitative interviews.
The in-depth semi-structured interviews were chosen because they support an approach of enquiry to explore understanding around complex processes and they can generate in-depth personal accounts . This interview method involves using open-ended questions around a topic with the use of an interview topic guide. The researcher can steer the direction of an interview but the interview format does not rely on a structured set of questions (Arthur, Mitchell, Lewis, & McNaughton Nicholls, 2013). The interviewee can be encouraged through probing techniques to give as much detail as possible within their responses to enhance exploration and explanation. The interview was an additional component to the research design to further explore in rich detail some of the issues raised in the questionnaire.

Section Summary
 The incorporation of a pre-tested, standardized tool is an important consideration when determining validity and reliability within questionnaire design.
 Questionnaire results can assist in identifying further areas for more in-depth exploration via the use of semi-structured interviews.
 Interview topic guides for semi-structured in-depth interviewsees can help steer the direction of the interview and ensure key subject areas are addressed, but they do not involve a set of specific questions.

Research Practicalities
From one sector of a health care Trust, all inpatient staff members from one assessment and admissions ward, five mental health treatment wards, and three residential rehabilitation units were invited to take part in the study. Participants were recruited through posters displayed within their working environment and through the distribution of questionnaires to their work area. Ten multi-disciplinary staff were selected to participate in the interviews. Interviewees were selected following participants' completion of a separate reply slip which was included at the end of the questionnaire. The selection of interviewees ensured a representative cross section of multi-disciplinary team roles. This can be referred to as a stratified sample which pertains to the sample being distributed in the same way as the population with regard to their criterion (Bryman, 2016).
To prepare for the main study and test the questionnaire design, a pilot of the questionnaire was conducted on eight staff from an inpatient forensic unit. Within this area, the multi-disciplinary staff also supported a patient group who experienced mental health problems and illicit substance use. A pilot of the semi-structured interview occurred with one of the respondents who had returned a slip from the questionnaire. The pilot interview tested that all main subject areas had been included in the interview guide and that effective structuring of the interview had been achieved. Bryman (2016) adds that piloting an interview schedule can also increase the interviewer's confidence by becoming more familiar with its use.
The research was designed and conducted as a component of an MSc degree and all aspects, including ethical issues, were fully discussed with the MSc supervisor. There were a number of key ethical considerations which were pertinent to this research study, which are outlined in the following.

The Role of "Insider Researcher"
As a health professional regularly encountering issues and care approaches to working with individuals with co-occurring mental health problems and illicit substance use, it was important to be acutely aware of my own values, judgments, and professional role when considering the proposed research. My clinical experience and own judgments had the potential to influence many aspects of the research and this needed to be acknowledged and accounted for.
First, I needed to consider that in the role of researcher, I may also have had relationships with participants as a manager or colleague and it was therefore important to support possible participants to freely decide whether they wanted to be a participant in the research without feeling obligated to do so. This was particularly relevant for one of the participating units in which I was the team manager. These issues were taken into account by ensuring the questionnaire had a separate removable slip to return, to indicate they would be interested in being an interviewee and I did not approach individuals personally. The slip was not identifiable with the questionnaire data. In addition, as my own experience and judgments around the subject area had the potential to influence the interpretation of data, the research was designed to include a peer review of the qualitative data to ensure that personal opinion and attitude were not influencing analysis and outcomes. Schwandt and Halpern (1988) refer to this process as an external audit to check and question whether inferences are logical and examine the degree of researcher bias.

The Voluntary and Informed Nature of Consent
The questionnaire format included a cover sheet which provided information on the purpose and process of the research as well as how much participant time would be required in its completion. The information also included how the data collected would be used and the possible benefits of the study. A statement was included to confirm that participation in the study was voluntary but completion and return of the questionnaire would indicate the participant's consent to take part in the research.
Interviewees were provided with an information sheet which explained issues of consent.
The main advice given included that the interviewee could terminate the interview at any time and could choose not to answer certain questions. For the tape recording of interviews, specific consent was requested, and it was clarified that the recording could be halted at any time by the interviewee. The interviewee was asked to sign a consent form stating that they understood the terms of the interview and research.

Anonymity and Confidentiality
Participants were informed via the questionnaire's introductory letter and the interview information sheet that anonymity would be guaranteed regarding their identity. For the semistructured interviews, participant identity was only known by the researcher (who was also the interviewer). Any comments or responses which may implicate the identity of the participant were avoided and reported in such a way that did not attribute their response with their identity.
Participants were informed that their confidentiality would be maintained or that if the researcher was concerned a response they had made could compromise their confidentiality or anonymity, then specific consent would be sought from the participant. Regarding data storage, data were stored and planned to be destroyed in accordance with the guidelines issued by the Trust Research and Development Department. Recorded interviews did not contain identifying labels on audio cassettes and hard data were stored in a locked cupboard on Trust premises. The participant identifiable data were not taken home to be transcribed. All of the above ethical issues were discussed and agreed by the Research Ethics Committee who gave favorable ethical opinion on the proposed research.
Section Summary  The role of "insider researcher" is important to consider with regard to possible bias processes which may occur during the research, and therefore, this needs to be a key ethical consideration. Having existing relationships with some participants has the potential to influence recruitment to the study, study design, and data analysis.  It is important that possible participants are given as much information as possible to support them in making an informed decision regarding whether they wish to take part in the study or not.
 A stratified sample was chosen to represent a cross section of professional roles.

Method in Action
Questionnaire Survey For the questionnaire component of the study, at the beginning stage, I found designing the questionnaire a complex process to ensure I was asking questions clearly and targeting the subject area so that could I could achieve a clear analysis of the results. After I had distributed the questionnaires, I found that not many were being completed and returned. I started to worry I would not achieve a good enough response rate. I had attempted to plan for this possibility by building in strategies such as informing participants via email they would be receiving a questionnaire, putting up posters advertising the research in each participating area, and recruiting a key staff member in each area to encourage participation and awareness of the research. The key staff member also had a role in distributing the questionnaires and I had informed them about the research to cascade to team members. I also visited each individual team to provide any further information. Despite these steps, from 270 questionnaires being distributed, a participant response rate of 36% was achieved. Mangione (1995) states that a response rate of below 50% is unacceptable; however, Bryman (2016) advises that the key consideration with regard to response rates is to recognize the possible limitations a low response rate may have on the specific research study and that in addition low response rates are common in many published research studies; 36% was a low response rate and disappointing for me, though it does align to other response rates of previous studies involving busy inpatient professionals (Mears et al., 2004;Tipper, Mountain, Lorimer, & McIntosh 2006). Additional steps I would utilize if using a questionnaire again will include paying close attention to devising clear participant instructions and ensuring a good questionnaire design and layout as these factors have been identified as improving response rate (Dillman, Smyth, & Christian, 2014). I would also pay close attention to generating interest at the beginning of the questionnaire by asking about issues which are more likely to generate immediate engagement.
The first part of the questionnaire had been designed to indicate from participant responses further improvements which could be made to perceptions of working with individuals who experience mental health problems and co-occurring illicit substance use. The majority of returned questionnaires were fully completed and the participant responses clearly indicated the staff viewpoints on the raised subject matters. The low response rate would not support the view that the study was fully representative of all staff members' views; however, the data collected could be utilized to build overall recommendations for practice development and indicate further research required in this area. In addition, although the data were obtained from inpatient units within one health care Trust, the Trust was similar in work and population to other Trusts and so may reflect similar issues experienced elsewhere.
The second component of the questionnaire which included the use of the Drug and Drug Problems Perception Questionnaire (DDPPQ; Watson et al., 2003) added validity to the questionnaire design as this was a tried and tested standardized tool used within other research projects. The DDPPQ identified staff perceptions toward drug users and the attitudinal scores showed to be comparable to the scores where the scale had been used in previous studies. In addition, it did, however, particularly highlight for my study that if training had been received by participants, this indicated a less negative attitude toward drug users than those staff who had not received training. This was a key result for this research study, and therefore, the use of incorporating the DDPPQ had been a good design step to consider and action.
With regard to data analysis of the questionnaire results, I used the SPSS package which I had never used before. With my MSc supervisor's guidance, I conducted statistical tests which I found very complex and unfamiliar. This was a very steep learning curve for me and in hindsight I wished I had undertaken some previous education and had some familiarity in the practice of using statistical software and appropriate statistical tests. I think this information may have helped with my questionnaire design too.

Interview Study
In contrast to the questionnaire component of the study, I did not have any difficulties recruiting to the number of interviewees needed for the study and I concluded that the information I had given to possible participants had been effective in allowing them to choose whether to participate or not. The challenges with conducting the interviews mainly involved time management factors. This included the practical aspects of conducting the interview and transcribing the interviews from tape recordings. An hours' worth of tape recorded interview took at least 3 hr to transcribe. This is a significant amount of time to factor in to the overall timetable of a study and should be considered with regard to the practicalities of fitting this in to a full-time clinical position and personal life factors. I would certainly consider the commitment needed for this part of the data extraction process when utilizing this method again.
For the data analysis process, I used thematic analysis, which involved an extremely indepth process of extracting themes and sub-themes and then asking a peer reviewer to comment on and review the methods I had employed and conclusions I had drawn. This again included a commitment to allowing enough time to ensure the thematic analysis was rigorous and any biases from myself could be fully explored with the peer reviewer. In hindsight, I think giving more time to the understanding of thematic analysis as a process would have been helpful in understanding the time commitment and knowledge needed to ensure I was completing the data analysis to the best of my abilities.

Section Summary
 The response rate to the questionnaire component of the study was low. Increasing the number of completed questionnaires in future studies would be an important area to address.
 Interview challenges included both the time allocation aspects to both transcribing data and ensuring robust thematic analysis.

Practical Lessons Learned Mixed Methods Research
Using a mixed methods study design enabled me to investigate my area of interest through the process of triangulation which Robson (1995) identifies as a means of testing one source of data against another. This successfully resulted in a more thorough investigation in exploring staff perceptions and experiences and I was able to compare and contrast the results of the quantitative and qualitative research methods used. There are data gained from the questionnaire survey and in-depth interviews which have through analysis overlapped and confirmed outcomes from both sets of results. In addition, some results contradicted each other and the individual methods used also offered unique results and observations.

Survey and Interview Study
By using two distinct methods which focused on gaining different forms of information, richer data were facilitated by yielding different kinds of data around the same subject and area of investigation. For example, regarding Training and Development, the questionnaire results showed that 40.5% of respondents reported receiving some form of training on illicit substance use and the key outcome of what respondents felt hindered them in working with illicit substance users was a lack in knowledge in how to work with these service users, as the majority had not received training. The interview data added further detail around experiences of participants also including the perceived need for additional training in looking at all aspects of working with illicit substance using inpatients, including the need for therapeutic interventions and legal and ethical considerations training. Henceforth, both the questionnaire and interview results had overlapping outcomes, which confirmed aspects of both methods' results. This sits well with an aim of triangulation proposed by Deacon, Bryman, and Fenton (1998), which involves a process of cross-checking findings, which result from quantitative and qualitative methods.

Contradicting or Conflicting Results
Another possible outcome of mixed methods research is obtaining contradicting or conflicting results from the methods used. For my research, developing care plans for users of illicit substances was a highly reported intervention for the questionnaire data (77%).
Considering this result, it could have been anticipated that interview participants would discuss the use of care plans within a range of favorable and positive themes confirming this perceived key intervention. In contrast to this, while interviewees viewed care plans as the main intervention directing care, they were viewed as problematic as were often not completed and hence created barriers to multi-disciplinary approaches and lack of clarity in therapeutic approaches toward the inpatient. A further subtheme identified there was a lack of multidisciplinary involvement in the care planning process. Contradicting and conflicting results in mixed methods research have been explained via the process of bringing different types of data together and making comparisons between them (Bryman, 2007) and that furthermore, the extent to which there is consistency in their findings can indicate the quality, validity, and credibility of the mixed methods study. However, Slonim-Nevo and Nevo (2009) advocates that a complementary approach accepts these conflicting findings by integrating them and highlighting the complexity of the subject which is being investigated, which in turn restores confidence in the consistency of the research. In addition, for my own study, I re-examined the design of my questionnaire and realized that I did not include any specific questions on the use of care plans, and if I had, this may have pointed to a problem area regarding their completion. These types of methodological problems have also been suggested as phenomena which may influence either contradictions or conflicting results within mixed methods research (Teddlie & Tashakkori, 2008).

Unique Results and Observations
The questionnaire captured specific information regarding interventions received by staff members as a result of a serious incident and ratings on how helpful the interventions had been.
The open-ended questions also gave specific information on the reasons why the police were being contacted over problematic situations. The interviews allowed in-depth exploration to occur in such areas as barriers to working with individuals with co-occurring mental health problems and illicit substance use which were explored through thematic categories. Interview data enabled the exploration of the practical and real experiences of participants which appeared to influence negative attitudes. However, underlying these accounts were staff explanations of struggling to deal with problematic issues such as drug dealing within inpatient areas and that they felt alone with managing these complexities.

Section Summary
 Using a mixed methods study design enabled research results to be confirmed through comparing data results through the differing methods utilized. In addition, contradicting and conflicting results from the differing methods were explored and unique results resulting from each method were identified.
 Interviewees described personal experiences which on the surface resembled negative attitudes but on further exploration accounted for struggling with problematic issues such as managing drug dealing within inpatient units.

Conclusion
Mixed methods research can facilitate a research approach which addresses a research question via a number of methods to gain a full investigation of the subject area. It can therefore involve innovative and creative processes which can spur the researcher on in the development of their research interests. However, the particular challenges of undertaking mixed methods research may entail both the development and application of knowledge in implementing the chosen research methods, which may be multiple and require a demonstration of competence in using quite diverse methods. Within the discussion of my own research, I have highlighted that this was a significant challenge for myself which I overcame via excellent supervision from my MSc supervisor, but in addition, it would have been helpful to also have some prior educational experience in utilizing the research methods and analyses of the research results. This may be a helpful consideration when contemplating using mixed methods research and devising a strategy on where research support, education, and supervision will be accessed.
However, that said mixed methods research can enable the researcher (with a supportive advisory arrangement in place) to gain rich knowledge in diverse methods and fully explore and analyze how differing methods may be applied to a research question to gain a wider or richer investigation of the subject area. For my own research, using mixed methods enabled statistical data to demonstrate key outcomes such as staff who had undertaken training in illicit substance use held less negative attitudes toward inpatients with these characteristics than staff who had not had training, and the length of time staff had been in practice did not affect attitude scores. The interview data then enabled a richer account of personal staff experience to be explored which provided a voice behind the statistical data and a personal powerful account of experience.
Throughout this case study, I have highlighted practical and logistical considerations I learnt along my research journey, such as in the future I would further consider strategies to increase questionnaire response rate and be more aware of time management challenges, for example, the transcription process and thematic analysis process from recorded interviews. Mixed methods research is definitely an approach to research I would use again as the methods employed can be tailor-made to suit the research area.
Section Summary  Using mixed methods in research can incorporate utilizing creative approaches to the area under investigation.
 Mixed methods research can include the need for rapid skills acquisition, and consideration of both educational and supervisory support is key to ensuring a highquality research study.
 Staff who had undertaken training in illicit substance use held less negative attitudes toward inpatients with these characteristics than staff who had not had training.
Classroom Discussion Questions 1. What attempts would you build in to your questionnaire survey design to encourage a higher response rate? 2. When formulating/choosing a questionnaire design, how would you ensure the quality of the tool with regard to reliability and validity?
3. What is the purpose of piloting a semi-structured interview?

What is triangulation in mixed methods research?
Multiple Choice Quiz Questions 1. Three disadvantages of self-administered postal questionnaires include a. Lower response rates, greater risk of missing data, difficult to ask a lot of questions.
b. Not convenient for respondents, they are expensive compared to interviews, they are slow to administer. c. They can be easily lost, probing questions can be asked, many respondents do not like questionnaires.
Correct answer: a 2. An interview guide is a. A collection of questions asked by the interviewer within a structured interview. b. A list of prompts and subject areas which is used when conducting either a semi-structured interview or unstructured interview. c. A document which is given to the interviewee to ask for their consent to be interviewed.
Correct answer: b 3. Informed consent refers to a. The participant signing a form to state they have read the participant information sheet.
b. The prospective participant being given as much information as possible to enable them to make an informed decision regarding whether they want to take part in a research study.
c. The participant telling the researcher they give their consent to be included in a research study.
Correct answer: b

Declaration of Conflicting Interests
The Author(s) declare(s) that there is no conflict of interest.