Collecting and analysing data to support Reflective Practice

Introduction

Being familiar with research methods may be useful to you in several ways. It can help you to understand and evaluate research published by others, or if you have students in your setting who require mentoring and support. Indeed, you may be undertaking your own studies. In this paper, we will present five research methods you can use to collect data in your setting to support Reflective Practice. As with many topics, research is a field in which the more you read, the more you realise it has its own language and acronyms, and there is always more for you to read and seek to understand. We aim to provide enough detail in this paper for you to both gather and analyse information/data. In addition, we highlight some of the dilemmas or ethical considerations you should be aware of when using these methods. The five methods covered are:

·       Journals

·       Questionnaires

·       Interviews

·       Observations

·       Focus Groups

 Getting Started

Once you have identified the area of work you wish to reflect upon, you can start to plan what information you need to help you. In the first instance, it is a good idea to look at what you already have available to you, for example, past observations or information from families and colleagues. You can then decide what you need to know more about to build a complete picture and be in a position where you can reflect upon the current situation and identify any areas you can develop or improve upon. Ask yourself:

·       what additional information you need to gather to understand the situation

·       whom you should include

·       what opportunities are available to you to collect information and to include those people.

Once you have chosen a method, you may feel that testing or piloting your preferred method(s) will help you to see if it provides you with the information you need. For example, you could practice an interview or a questionnaire on a colleague. Experienced researchers do this as it is far better to discover that people do not clearly understand what you are asking at this stage than after you have questioned 10 people.

Some other factors to consider as you set out your plan are how you will conduct yourself and how you will inform others about what you are doing. You must conduct yourself in a professional manner, and you must be clear with anyone involved that you are collecting data and treat that data as you would any confidential information you hold on any family or child in your setting. It may seem harmless to inform a colleague about what another has said during an interview, but you do not want people to feel they cannot be honest with you. If you plan to share information, this is fine, but you must tell all those involved before you collect the information that this is what you plan to do. You must gain their consent and accept that they may, in some instances, decline the invitation to participate.  

Five Methods for Gathering and Analysing Data

Journal

Gathering Data: A Research Journal or a Reflective Journal can be a notebook or a file on your computer to store your notes. It can be advantageous as you can keep a record of details and your thoughts. You may not realise at the time that something is important.

For example, one researcher reported, ‘I was informed a child in a setting I was observing was having a challenging time at home as her parents had separated. I recorded this in my journal. It was only when I analysed the observations weeks later that it became apparent that this coincided with changes in her behaviour’.

Analysing Data: As a journal records your thoughts, there is little to do to analyse the data. However, it can be useful to provide you with a timeline, reminding you of events and giving insight. For example, you may record in your journal:

‘Izzy's mum came into the setting today to advise us that she and her husband had separated and that Izzy had been displaying some challenging behaviours at home.' 23^rd June 2019.

When reviewing your observations, you have noted that there has been a substantial increase in the number of challenging behaviours Izzy has displayed.  You may then review the information you gathered during interviews and find that colleagues have also commented on this:

 ‘…we have seen a deterioration in Izzy's behaviour, which I believe is a direct result of the challenges she is facing at home. As we have seen her interactions with her peers become more aggressive, we have also seen an increase in the demands she is placing on our time; I think she is demanding our attention’. Practitioner 1, Interview 2

Together these three different sources of information provide you with an insight. It is not merely an opinion that there has been an increase in challenging behaviours, but you also have a record of this increase. Your diary noted a possible cause, and this is backed up by a practitioner in your setting. You now have enough information to draw a conclusion and begin to reflect upon what further information you may need and what can be done to support Izzy. As a practitioner you will be making decisions like this all the time. Reflective Practice can give a structure to this, can highlight issues which are less obvious and may fall through the cracks and provide you with clear evidence for your assertions.                                                                                                                                                                                                                                                                                

Ethical Considerations/Dilemmas: The ethical concerns of keeping a journal are minimal. Anybody can record their thoughts. It is wise, however, to consider how secure the information is, can it be opened and read by others? You must take precautions to keep information private. You can achieve this by keeping the data in a locked or password-protected file,  by not keeping any sensitive data (check your policy on data protection for help with this) and by using codes. For example, here we have recorded the child's name as Izzy; this is a pseudonym, it is not her real name. We have also recorded the practitioner as Practitioner 1, not by their name. This decreases the likelihood of anybody reading and understanding your notes. Of course, sometimes you need to use names, and this is often appropriate, but be mindful of times when it is not.

Questionnaires

Gathering Data: Questionnaires can be a great way to gather information from families and colleagues.

Some of the advantages of using questionnaires include that:

·       they can be more objective than other methods as everybody receives the same information

·      you can gain information quickly

·       you can include more people than perhaps you could for interviews as they take less time

·      the participants may prefer them as they can respond in their own time rather than having to commit to an interview

·      they can be completed anonymously.

 There are also some disadvantages to using questionnaires, including that:

·       response rates are often low

·       it is not possible to explain any points in the questions that participants might misunderstand

·       open-ended questions can generate large amounts of data which you may not have time to review

·       participants may answer superficially, especially if the questionnaire takes a long time to complete.

Start with a flow chart; what do you want to find out more about? Where do you need to start, and how does one question lead to another? You must be careful not to use acronyms and terms that some people may not understand or are ambiguous, e.g. what does ‘regular' mean?

There are different ways of asking questions on a questionnaire. Consider which will provide you with the information you need. Do you wish to use open questions, closed questions, or a combination of these? We have provided some examples below to help you consider which option would be the most appropriate for your work. You may wish to use a combination.

An open question:

At the moment we offer three attendance options: morning sessions 8am-12.30pm, afternoon sessions 1pm-5.30pm and all-day sessions 8am-5.30pm. Would you like to see any changes to our current session times? What changes would you like to see?

A closed question:

At the moment we offer three attendance options: morning sessions 8am-12.30pm, afternoon sessions 1pm-5.30pm and all-day sessions 8am-5.30pm. Would you like to see any of the following changes to our current session times?

o   Drop off available from 7am

o   School day option 9am-3pm

o   Late collection available to 7.30pm

o   Overnight care from 7pm-7am.

You can include ‘Other, please specify’ at the end of a closed question. Think about what will be helpful to you and if you will have time to look through all the answers. 

Another way of asking questions is to use Rating Scales. You could ask, for example:

I would use overnight care if it was available in the nursery?

1 = Strongly Disagree

2 = Disagree

3 = Neutral/Unsure

4 = Agree

5 = Strongly Agree

Analysing Data: Before you can draw conclusions from the questionnaires, you must organise the data. Look through the responses and check that the questions have been answered as you expected? Do any need to be omitted because they have been misunderstood?

You may find that you have two different types of data: Quantitative Data and Qualitative Data. Qualitative Data is data that records behaviours, reported feelings and behaviours and seeks to interpret and to understand, for example, observing how well children in your setting play together. Quantitative Data is concerned with gathering numbers and figures, for example, the number of behaviour incidents recorded over a specific period of time.

You are unlikely to have any volume of Quantitative Data. You can tally up the numbers and record these as required. For example,

·       How many respondents?

·       Any information you collected regarding age, sex, job role. Collate these into numbers, i.e., 13 males, 15 females.

·       For closed questions, record how many respondents selected each answer, i.e., 5 parents said they would like us to offer school day hours 9-3pm.

·       You can present the numbers in a table or in sentences.

You may have read or seen people presenting numbers and figures as bar charts and pie charts, percentages or possibly conducting complex analysis. This is not necessary with small quantities. Percentages can be particularly misleading with small numbers. For example, you could say 25% of parents stated they would use overnight care. This sounds very important until you see that there were only eight responses and two said they would use the service. 

With the Qualitative Data you have collected, you can spend some time getting to know the data. Reading it through and seeing what strikes you, what relates to the literature or other research you have read. You can search for themes. You can use computer files, post-it notes, or highlighter pens to identify your themes to group information so you can review it later. For example, you may note that four parents and two practitioners commented on chaotic drop off times with everybody arriving at the same time. You could highlight all these in yellow or copy and paste the relevant sentences into a word file. This will help you to see how many times an issue comes up and looking at these together allows you to reflect on what the information is revealing.

You can review these themes in relation to relevant literature, data you have collected from other sources or note that there is a need to find out more. For example, if three practitioners said they needed more support to provide for a new child with a visual impairment, you can reflect upon what you may be able to offer and decide to seek information from local agencies.  

Ethical Considerations/Dilemmas: As with all data you must be mindful of where it is stored, who could have access and how you will protect those involved and keep their data safe. If you plan to use quotations from questionnaires in reporting, ask yourself if you have permission to do so and if others can identify the person you are quoting. It is better to ask and to check than to divulge information given to you in confidence. 

Interviews

Gathering Data: Interviewing colleagues, families and children can provide you with a lot of useful information and insight. As with the questionnaire planning, take some time to develop a chart of what you want to find out, the questions which can help you to achieve this and whom you need to involve. Some of the advantages of interviews include:

·       that you can clarify questions in the moment in a way you cannot with a questionnaire

·       You can also add questions to gain deeper insight into interesting responses.

However, there are some potential disadvantages which, given the limited time and resource you are likely to have, should be carefully considered. This includes that:

·       They are time-consuming

·       You can impact on the responses you receive if you are not consistent with the questions you ask and how you ask them

·       There is a need to consider your prior relationship and whether they feel comfortable and confident in talking to you. You may have to reassure those who agree to an interview that you are interested in the responses, that they will be treated with respect and that their information will be kept confidential unless otherwise agreed.

There are several different approaches you can take to interviewing. We provide a summary of the benefits below:

Face-to-face Interviews

·       You can develop a rapport with the respondent

·       You can monitor body language

·       You can delve deeply into issues

·       You can speak ‘off the record’.

Telephone Interviews

·       You can gain access to participants who cannot attend a face-to-face interview. Perhaps a parent or carer can speak with you on the telephone for 10 minutes but would be unable to come to the setting to meet  with you

·       They can take less time, and you can, for example, call people one after another rather than see people in, seat them, etc. They can, therefore, be less costly.

Informal Conversations

·       It is likely you engage in many informal conversations in your setting which are useful in providing information and insight

·       This is beneficial as they aren’t time consuming and fit into the natural pattern of the day

·       From a research perspective, it is essential to be mindful that these conversations may only provide information that can be shared in the moment, possibly in front of others and that we can get into habits of speaking to each other in particular ways. The meaning is, therefore, not always reliable. 

Analysing Data: As with the questionnaires, look through your data and search for recurring themes and points of interest. Can you interpret these? Do you have data and information from other sources that can help you to Reflect and build your understanding? 

Ethical Considerations/Dilemmas: The ethical considerations on the storing of data and the reporting of data are the same for Interviews as they were with the other methods. Store them securely and get permission before using quotations. With Informal conversations, a source you may often use to gather information for your Reflections, being mindful of ethical considerations is particularly important. Colleagues and families may not understand that what they have said to you may be included in any reporting you do. It is important to be clear about this.

Observations

You will no doubt have experience at conducting observations. You will likely have guidance in your setting on how they ought to be undertaken and recorded. That said, do spend a little time considering whether you need to add any additional categories or details which will support your Reflections. Some possible things to think about include:

·       What you will look at – specific, times, behaviours or individuals?

·       Where and when will they be conducted? When are you likely to see things that will support your Reflections?

·       Where will you record the information? Blank sheets, pre-prepared forms?

·       Will you verify/discuss your observations with anybody else and gain their opinion?

Conducting observations provides you with an opportunity to hone and develop the skills of Reflective Practice. As a Reflective Practitioner you need to go beyond the impressionistic and try not to allow your previous experiences to determine what you see and what you record. Try to be open to all that is happening and be open to new interpretations as to why. 

Analysis of Data: As with the previous methods, you need to read through your data and become familiar with it. If you have collected data that provides quantitative data/numbers, record these, then look at the more descriptive data, the qualitative data and look for themes and events that stand out. You can then reflect on these alone or with a colleague. This will develop your Reflex skills, by listening to and seeking to understand another's interpretation of what you recorded.

Ethical Considerations/Dilemmas: The usual rules of storing data securely and being mindful of how you report your findings apply here. There is also a need to consider your impact on the situations you observe. You may feel that the children in question are so used to you conducting observations that it does not impact their behaviour, but ask yourself: would they potentially have behaved differently? Are your colleagues behaving differently as they know you are recording your observations? How may this have impacted upon the situation? This may be particularly important in some instances, for example if you are looking to see why a session you planned has not been going well, only to discover that when you observe the session it is running smoothly.

Focus Groups

Focus groups are much like interviews, but you have a group of people rather than an individual. Some of the benefits of Focus Groups include that:

·       You can gather information from a larger group of people in a short time, for example, at the end of a staff meeting

·       You can gather rich data as individuals discuss issues with each other in detail

·       You may find points of interest and value that you may not have thought to ask about as you have less control over the discussion

·       You can explore new areas without having to set a series of pre-planned questions, i.e., you can ask for opinions on a specific topic and see what is unveiled.

There are some challenges and disadvantages to conducting focus groups, including that:

·       You are relying on the group to interact

·       Participants may steer the discussion in directions not relevant to your reflection and research

·       Some participants may dominate the discussion

·       Ethical issues, e.g. there is little confidentiality as everything is said in front of other participants.

There are some things to consider when you are planning to run a group interview or Focus group, including:

·       Taking time to establish the rules for the discussion, e.g. one person speaks at a time; everyone to participate, no-one to dominate; everything said is confidential

·       How you will record the information – it is difficult to write notes when a group are talking?

·       How and when will you steer a discussion back to your topic of interest?

Data Analysis: The advice for analysis is similar to that for interviews, but there are key considerations. This includes whether you will be able to distinguish between what was said by whom, and whether the group situation impacted upon what people were willing to say.

Ethical Considerations/Dilemmas: All of the issues we have raised for the previous research tools apply here. In addition, you have to consider whether the views that were expressed are a true representation because they were expressed in a group. Are people swayed by others or have they omitted details? Most important from an ethical perspective is to try to ensure the group understands that what has been said must not be repeated outside of the group without prior permission.

Conclusion

This paper provides the information required to use research methods to gather data in your setting. These methods may sit alongside other ways in which you collect information. This information is only useful in this context if you then take the time to:

·       understand what the analysis has revealed

·       consider your role and your approach

·       reflect on what you can learn from this.

How can you then use this learning to make a change? Reflecting on the information you can consider:

·       what conclusions you can draw about the challenges and successes?

·       What you could build upon, what could you change?

·       How will you go about making change?

·       How and when will you review any changes?

Only once you have sought to understand the information, considered the lessons to be learned, and planned and implemented change, does this process of data collection become a part of being a Reflective Practitioner.

This is a companion piece to Jen's article on the Reflective Process.

Recommended Readings

Brown, Z. & Perkins, H. (2019) Using Innovative Methods in Early Years, Routledge, Oxon.

Mukheri, P. & Albon, D. (2018) Research Methods in Early Childhood, Sage, London.

Pollard, A. & Contributors (2019) Techniques of Enquiry, available at:

http://reflectiveteaching.co.uk/books-and-resources/rts5/part1/developing-an-evidence-informed-classroom/

Robert-Homes, G. (2018) Doing Your Early Years Research Project: A Step by Step Guide, Sage: London.