Skip to start of content

HomeReportsFamily violence → Methodology

CRC funded reports

Methodology

published in:
Family violence : young people and youth sector workers, informing government about the implementation of mandatory reporting in Victoria
Danny Sandor and Julian Bondy
ISBN 07311 0575 3
Fitzroy, Vic. : Youth Affairs Council of Victoria, 1995
Criminology Research Council grant ; (15/93-4)
October 1995

In this section the two major research exercises will be described, followed by some remarks about the limitations of the methodology.

1. The Project Steering Committee

A project steering committee was established to advise the researchers in the operationalisation of the research. Its major roles were to assist in the research time plan, to support and provide advice in the questionnaire design and protocols for the focus groups, to assist in the provision of research sample and to assist in the selection of peer researchers. The steering committee comprised of representatives and key players in the youth welfare, juvenile justice and youth affairs sector. Membership included:

2. Consultations with young people

The consultations were designed to identify the key interests of potential consumers of the mandatory reporting system, that is, adolescent young people. The consultations with young people were focussed on the following four areas:

The Sample

A total of twenty five focus groups with young people were conducted, including two pilot groups. Each focus group aimed to comprise eight to twelve young people for optimum participation (Schatzman and Strauss 1973) but this was not always achieved. The total sample of participants comprised 163 young people.

Young people aged between twelve and sixteen years of age were invited to participate in the focus groups. Mindful of the sensitive nature of the areas under discussion and the ages of the respondents, informed consent was sought prior to any engagement. Participants were given a consent form which was explained to the prior to the commencement of the group (See Appendix 1 for a copy of the consent form).

Participants were paid $12 for their attendance. The consultations were organised in two sample groups:

Some of the generic youth services groups were found to contain young people with first-hand experience of the protective services system or friends/acquaintances with such experience.

A further consideration which guided the intended sample was the need to balance statewide demographic and geographic characteristics with the different child abuse notification rates in various areas around Victoria. For instance, locations were sought in white, and blue collar areas, with a bias towards areas with higher notification rates.

The intended sample comprised 8 out of the 25 groups being conducted in rural settings as approximately 30% of Victorian young people aged 15 to 19 living outside the Melbourne and Geelong metropolitan areas (Australian Bureau of Statistics, 1986 Census Data).

Groups were conducted in both regional centres and smaller towns.

The H & CS system is organised and administered in 4 metropolitan and 3 rural regions. It was intended to conduct at least two groups in each of the five metropolitan and four rural H & CS regions, with an extra group being conducted in regions with the higher reporting ratios per person wherever possible.

Using maps showing regional boundaries, available demographic statistics and H & CS reporting data, a set of target municipalities was selected. The youth services coordinator from selected municipalities (where applicable) was approached, and the viability of conducting a general focus group in their municipality was discussed. A set of information about the project was then sent to workers who agreed to assist the research by organising a general focus group consisting of eight to twelve young people.

Under advice from the steering committee, it was the intention to avoid using H & CS services to assist in organising targeted focus groups. There was concern that participants may not be seen to attend groups on a purely voluntary basis if they were organised by statutory workers. When local agencies were contacted to assist in organising targeted focus groups, however, there was general reluctance on numerous grounds. It was therefore necessary to engage the department to request local H & CS services to assist in organising the targeted groups.

The process of organising and conducting these groups was a sensitive exercise. Inaccurate verbal information provided to young people from workers, instead of the intended information sheets, at times fostered misconceptions among the participants about the study's aims and methods. This confusion jeopardised the viability of a number of focus groups, and made it necessary to run one 'mixed focus group' with young men and young women together. One single interview was also included in the data set. A young woman from a rural area, who had experience with protective services was interviewed. She was the solitary participant in a targeted female group originally expected to consist of six young women.

Focus group design - peer research

The focus group method is gaining ground as an appropriate method of consulting with young people regarding their experiences and perceptions of social conditions (see for example Australian Youth Foundation 1993). Focus groups use structured discussion in a group to elicit qualitative information. Given the exploratory nature of the objectives of this study it was considered most appropriate to consult with a large number of young people in groups, rather than pursuing a smaller sample through in depth interviews. These focus groups discussions were recorded on tape for subsesequent transcription and detailed analysis.

The development of the discussion structure was pursued in three particular stages:

Employing young people as consultants and researchers was a major aspect of this study. Peer research methods are premised on the assumption that the research subjects provide more extensive and reliable information to those with whom they feel more culturally attuned and that young people are more comfortable in interviews conducted by young people who have had related or similar experiences. (Wilson and Arnold 1986)

A methodology which relies on the use of peer researchers conducting focus groups will necessarily contain instances where the form of questioning or prompting is framed in a manner less 'pure' than one would see in a highly structured questionnaire controlled by professional researchers. This is a trade-off which we sought to minimise through the training of the peer researchers and the presence of the adult researcher at the groups and we consider the rich and candid material we have obtained reflects the adequacy of these measures.

The peer research approach has been used in numerous projects on a range of different topics in recent years (Ferguson 1993; Knoblach 1992; Moore & Rosenthal 1992; Robson 1992; Alder and Sandor 1989) with favourable results and published discussions of their experiences in involving young people in their projects (Hill 1994; Wilkins et al 1993; Alder & Sandor 1989).

For the present study, a number of youth sector workers in metropolitan and rural settings were approached to refer young people who were thought potentially appropriate as peer researchers to the project. These young people were informally interviewed by the researcher and talked through an initial discussion paper regarding the design of the focus groups. These young people were paid $20 for their contributions to the design of the focus group discussion structure. A short list of young people were then invited to be interviewed by the researcher, a principal investigator and a member of the steering committee. It was attempted to make these interviews as informal as possible.

A formal approach to the recruitment of peer researchers in this project was pursued for two reasons. Firstly, the nature of the study as a statewide consultation project required a rigorous approach to recruitment in order to incorporate regional and cultural diversity; secondly, conducting focus groups required a level of pre-existing skills in addition to relevant personal experience.

Focus groups: the process

In recognition of the diverse experiences and interests of the sample group, the final discussion structure consisted of four separate stages, reflecting a variety of response modes, ie, informal responses, formal responses and recommendations. This structure was adapted according to the characteristics of the different focus groups. This flexibility was necessary due to the great variation in participants' backgrounds and personal experiences. In order to obtain recommendations regarding mandatory reporting it was also neccesary to engage in some education about mandatory reporting and the intial investigation of official reports of child abuse. The assessment of young people's pre-existing knowledge came before these later parts of the focus group.

The first stage of focus groups involved completing the consent form which ensures the anonymity of participants. The consent form was signed by both participants and researchers. (See Appendix 1).

Participants were told that they were being consulted regarding the implementation of mandatory reporting laws. Introductions and where appropriate, "ice breakers" were conducted before turning on the tape recorder. Feedback was then invited regarding participants' knowledge of mandatory reporting laws and the child protection system. Facilitators then provided information about the new law, and invited comments on the investigatory process which were recorded on a white board where available. A break for refreshments usually occurred following this stage.

The break was a critical component of the group structure as it allowed young people to chat informally with the tape recorder off. Any initial trepidation about the group had usually dispersed by this stage, and participants were usually quite relaxed for the second half of the focus group.

The third stage involved using three scenarios to promote discussion of a third party incident, rather than focussing on participants' individual experiences. Participants were specifically asked whether they would recommend a report to protective services for each abusive scenario, and to explain their answers.

In the final stage, the facilitators reflected on the prior discussion in the group and invited recommendations about the new laws. The recommendation themes provided by participants in the initial groups were then presented in subsequent groups for further discussion (see Appendices 2, 3 and 4 for further information).

The general sample groups were joint facilitated by the two peer researchers, with the adult researcher attending groups at times to provide feedback on the performance of the peer researchers. At times the researcher became actively involved in focus groups, particularly when identifying young people's recommendations. The female targeted focus group was facilitated by the female peer researcher and male groups by the male peer researcher with the researcher in attendance.

The duration of focus groups was 1.5 to 2 hours. All participants were paid $12 for their attendance.

Focus group data analysis

The task of the researcher in qualitative research is create and order ideas by classificatory categories in unstructured records. However, the categorising of material from the same topic is not an end in itself. The aim in qualitative research is the discovery and ordering of themes and ideas through the emergent 'stories' and patterns.

The Richards, in their paper on the theory of categories and how they may be exploited by computational qualitative data analysis urge researchers to derive and use categories in two ways; firstly to develop data driven categories, that is that the formulation of categories come from the data and are not imposed prior to data collection. Secondly, to treat the categories as being linked and structured so that category production and the use of categories become the method of theory construction. (Richards & Richards 1994:1)

In concordance with these recommendations, theory in this part of the research was developed which was 'grounded' in the reality being studied rather than the traditional deductive process such as a priori hypothesis formulation. The advantages of this open approach over a pre-structured study were that by allowing substantive concepts and hypotheses to emerge on their own, the researchers were less likely to screen out relevant concepts and hypotheses. The storing and clarifying of our developing understandings and the linking of these ideas to data allowed formal theory to enter after the researchers had become convinced of its relevance.

All focus groups were taped and transcribed. Identifying remarks and extended passages of information provision from the researchers were not transcribed. After being transcribed, the peer researchers listened to the tapes again, and where possible they completed passages obscured by background noise.

The transcriptions were then transferred into an appropriate computer text format and entered into a qualitative data analysis program as a whole document. The N.U.D.I.S.T. program was selected as it allows great flexibility in manipulation, and detailed analysis of a large data set. The complete set of data amounted to approximately 200,000 words. The researcher, in conjunction with principal researchers developed a 'conceptual tree' for analysis of the data. This tree was a conceptual framework reflecting emergent themes in focus groups. Passages of text were indexed at multiple nodes reflecting whether statements reflected for example, a perception, a statement about knowledge, an experience, or a recommendation regarding a set of agents. By attaching various codes of this kind, text searches and inquires were conducted to investigate the frequency, and characteristics of the 'intersections' and 'unions' of the various nodes. (See Appendix 5 for the conceptual tree)

Methodological constraints

The findings from the major research tasks provided a set of results which sufficiently fulfill 0the stated objectives of the study. It is however worthwhile to comment on a number of interesting aspects of this study. The broad objectives and logistical constraints ensured a general aproach to the question of how mandatory reporting is perceived by youth sector workers and adolescents. For instance, the broader theme often obscured important detail around the different cultural perceptions of abuse, and appropriate responses to abuse.

A recent NSW study has identified through consultations, the range of experiences and concerns relating to child and adolescent protection of refugee and/or migrant young people (NSW Child Protection Council 1993). Attempts were made to identify the different interests of non-English speaking background (NESB) groups, however, due to logistical constraints this was only partially achieved. The characteristics of the majority of youth services involved in this project reflect the dominant anglo-celtic cultural tradition.

It was considered innapropriate to pursue focus groups with any specific NESB groups as this would have risked the sample suggesting that chosen NESB groups were at increased risk of abuse. In groups where a proportion of young people were from a NESB, it was difficult to prompt discussion along cultural issues due the sensitive nature of the groups. The research team reported that, at times, certain interesting topics arose but were not pursued in order to maintain the priority of a supportive and non-confronting group style. For NESB participants, it was difficult to pursue their perceptions of cultural issues in ways which allowed them to feel comfortable, and not highlighted as 'special' in the context of the rest of the group.

A similar limitation to the study concerns the absence of the particular views and experiences of Aboriginal and Torres Strait Islander children who are greatly over-represented in notifications of abuse (see Rayner 1995:45-48; as to Judicial recognition of the special status which these peoples warrant in evidentiary and other senses, see the decision of the Full Court of the Family Court of Australia in B and R and Separate Representative, 27 September 1995 at Melbourne, as yet unreported).

One further limitation of this methodology was the reliance on the goodwill of youth sector workers to invest time and energy in organising focus groups for the research team. This obstacle took different dimensions for the general, and targeted focus groups. For the general focus groups, there was concern from workers regarding the sensitive topic of discussion. Information packs were sent to workers which included general information about the project, and an information sheet for participants and interested parents. Despite these efforts, the research team found that often participants had not been provided with the information sheets provided, or had been given misleading information about the project.

As mentioned earlier, it became neccessary to approach H & CS to assist in organising the targeted focus groups. Again, the research team found that misleading information had been passed on to the participants. This caused confusion and young people mentioned that they had been anxious about the focus groups, and some of their friends had decided not to attend, due to misinformation. Having said this, it was often very difficult for workers to organise enough young people at the specific time of the focus group, due largely to the transient nature of many young people in care, and the day to day issues and activities of the participating agencies.

The final caveats concern the nature of data collected in focus groups.

First, while attempting to ensure otherwise, we accept that there is a possibility that not all of the references were free from the possible fact that participants were influenced by the study having been conducted under the co-auspice of the peak body representing youth sector workers and youth issues in this State. We doubt this is so but for the sake of completeness we advise that only replication under other conditions could test this.

Secondly, it must be understood that we have collected data about young people's understandings and perceptions. This research reports how young people viewed the issues attendant to mandatory reporting and apects of the protective system. The study takes their claims as truly held perceptions, however, the research does not purport that the participants' views are "objectively" accurate nor that the data are representative. We are however confident that the research has captured what young people say about the issues in question.

3. Survey of Workers with Young People

The sample for the questionnaire was the membership of the Youth Affairs Council of Victoria ("YACVic"), the peak body representing the youth sector in Victoria. This sample group consists of workers in the categories of local government, emergency accommodation providers, and community outreach projects (Youth Affairs Council of Victoria 1993 : 35-38). A total of 383 questionnaires were distributed through mailout to YACVic members, and 219 through youth work network meetings throughout Victoria in May and June of 1994. 150 surveys were returned representing a 24.9% return rate.

Questionnaire design

The questionnaire was designed to obtain responses from workers in the youth affairs field along the following themes:

The final objective was for the findings from this survey to contribute to a targeted training package for youth sector workers, based on the final 'competency units' derived by the National Youth Sector Training Unit's Core Competencies Project.

There is a paucity of previous work regarding training need analysis using questionnaires, particularly in the youth affairs field. Previous surveys in the area of training need analysis were either designed as part of training course evaluation, or were too general to be of assistance in this study.

By examining the methodological models used in other research examining mandating legislation and the reporting behaviour of professionals and the methodological models used in research regarding youth work and working with young people who have been abused, the questionnaire was roughly conceived. Further design modification took place after consultation with youth sector workers and a pilot questionnaire which was sent by mail to ten randomly selected workers. Another seven were distributed at a network meeting. These were not included in the final data-set. Upon the return of these pilot questionnaires and consultation with the steering committee the questionnaire was modified, finalised and distributed.

A questionnaire design was sought which provided opportunities for both formal, and extended responses. Questions were framed in language designed to avoid response sets.

Items were The questionnaire was divided into five sections, which will each be explained in turn (see Appendix 5 for the final survey).

Section one covered the demographic profile of respondents, the type of service and the duties performed by respondents, the qualifications of respondents, the work experience of respondents, and several questions designed to enable a comparison of the sample with the known characteristics of the youth sector worker population. These were to be used to determine if there were any particular groups of youth sector workers who had substantially different skills, attitudes or knowledge regarding mandatory reporting compared to other youth sector workers. These questions were all multiple choice.

Section two asked workers a series of questions designed to test their levels of knowledge about mandatory reporting and what their sources of this knowledge were. The first part dealt with respondents' knowledge of mandatory reporting, and the other key laws relating to child and adolescent protection. A set of 14 statements were developed which covered the minimum legal knowledge required to work effectively with young people involved in the protective system. Respondents were instructed to indicate whether a particular statement was true, or false, or whether they were unsure. The second part asked respondents to indicate if they had received training in 11 areas relevant to working with young people who have been abused, and to indicate the source of this training. Part 3 simply asked respondents to indicate where they had heard of mandatory reporting legislation. These questions were all multiple choice.

Section three asked youth sector workers to indicate what areas of knowledge they regarded as important and how they rated their skills in relation to those areas. Whereas section 2 dealt with the knowledge of workers and a range of specific training subjects, section 3 attended to the broader concept of skill, upon which the 'core competency units' are based. The traditional concept of skill, however was extended to include workers' perceptions regarding the relevance of workers' attitudes and beliefs to practice with young people. Using a recent training needs analysis kit as a template for this section (Archer 1991), workers were asked to indicate their perception of the importance of each of the 22 skill areas in working with young people who had been abused and then to assess their level of competence in these skill areas. There was also an opportunity for workers to mention other skill areas not included in the multiple choice section. Each skill area began with an action verb and fell into 4 main areas:

Section four identified workers' experience in receiving disclosures, and asked them to provide short answer data regarding their attitudes towards taking action on such disclosures, and explored their perceptions of mandatory reporting laws and the impact of the laws on their relationships with young people. The first part asked workers to estimate the number of disclosures of physical and sexual abuse received from young people aged 12 to 16 in their work with young people. The second part asked respondents to comment on the circumstances surrounding reports to the authorities, and instances where the department was not notified. The third part provided a number of statements regarding mandatory reporting laws which are characteristic of the range of attitudes towards mandatory reporting and asked workers to indicate their level of agreement with these statements. The specific themes of these statements were generated from the literature search, in consultation with workers and the steering committee.

We note here that a limitation of the survey was its focus upon workers' self-report treatment of disclosure. There was no attempt to investigate the statements of our sample with their actual past behaviour. Secondly, we focussed deliberately upon their treatment of disclosures rather than their actions upon forming an assessment of likely or actual abuse without having been approached by a young person with a disclosure.

Section five prompted workers to provide recommendations for the implementation of mandatory reporting and the training needs of workers. After providing a summary explanation of mandatory reporting laws, respondents were then given five specific prompts to focus their short answer responses. Workers were then provided one final opportunity to make any further comments about mandatory reporting laws and any further feedback regarding the survey itself.

Methodological Constraints

According to the 1995 Youth Worker census and training needs survey undertaken by YACVic on behalf on the Ministerial Review of Youth Worker Training (YACVic 1995), there are 12,314 workers in 954 services who work with young people who fall within the rubric of 'youth worker' in Victoria. Of these 4355 are paid workers of whom 1,887 (15.3%) are full-time, 1,029 (8%) are part-time and 1,439 (11.7%) are casual employees According to this census there are also 7,959 (64.6%) voluntary workers. One result of such high numbers of voluntary, casual and part-time workers are low levels of occupational identity. This is reflected in the low levels of returns. Because of the low number of returns to questionnaires sent out caution must be taken in treating the sample as representative of youth workers in Victoria but, as the reader will see, we have tested our sample against various features of the census data in reporting the results of the survey.

Consultations with Workers with Young People

The consultations were designed to provide mechanisms whereby workers with young people would be provided with information generated by the research and opportunities to comment on the draft findings. Two methods of formal consultation were engaged. Firstly the Steering Committee included representatives from youth work agencies and youth sector workers Their comments have been incorporated in the interpretation of the youth sector survey presented herein.

Secondly the Mandatory Reporting and Youth Work Forum was held in September 1994 and was attended by approximately sixty people. Speakers included: both Principal Co-Investigators, Maria Varkopolous - Footscray Youth Housing Project, members of the Steering Committee; Greg Levine, Senior Magistrate - Children's Court, Colleen Clark, Manager - Community and Professional Educational Unit, Protective Services for Children and Adolescents, Health & Community Services, and Mark Longmuir, Executive Officer - Youth Affairs Council of Victoria.

After the presentations participants split into workshops in three thematic areas:

If you see this message you are probably using an old browser: these pages should be readable, but we recommend updating to a modern browser.