Published on in Vol 4, No 2 (2015): Apr-Jun

Development of a Culturally Appropriate Bilingual Electronic App About Hepatitis B for Indigenous Australians: Towards Shared Understandings

Development of a Culturally Appropriate Bilingual Electronic App About Hepatitis B for Indigenous Australians: Towards Shared Understandings

Development of a Culturally Appropriate Bilingual Electronic App About Hepatitis B for Indigenous Australians: Towards Shared Understandings

Original Paper

1Menzies School of Health Research, Global and Tropical Health, Charles Darwin University, Darwin, Australia

2Royal Darwin Hospital, Infectious Diseases Department, Division of Medicine, Darwin, Australia

3Miwatj Health Aboriginal Corporation, East Arnhem Land, Australia

4Royal Darwin Hospital, Chronic Disease Co-ordination Unit, Division of Medicine, Darwin, Australia

5Dreamedia, Darwin, Australia

6Centre for Disease Control, Department of Health, Darwin, Australia

Corresponding Author:

Jane Davies, MBBS, DTM&H, MRCP

Menzies School of Health Research

Global and Tropical Health

Charles Darwin University

PO Box 41096


Darwin, 0811


Phone: 61 889468640

Fax:61 89275187


Background: Hepatitis B is endemic in Indigenous communities in Northern Australia; however, there is a lack of culturally appropriate educational tools. Health care workers and educators in this setting have voiced a desire for visual, interactive tools in local languages. Mobile phones are increasingly used and available in remote Indigenous communities. In this context, we identified the need for a tablet-based health education app about hepatitis B, developed in partnership with an Australian remote Indigenous community.

Objective: To develop a culturally appropriate bilingual app about hepatitis B for Indigenous Australians in Arnhem Land using a participatory action research (PAR) framework.

Methods: This project was a partnership between the Menzies School of Health Research, Miwatj Aboriginal Health Corporation, Royal Darwin Hospital Liver Clinic, and Dreamedia Darwin. We have previously published a qualitative study that identified major knowledge gaps about hepatitis B in this community, and suggested that a tablet-based app would be an appropriate and popular tool to improve this knowledge. The process of developing the app was based on PAR principles, particularly ongoing consultation, evaluation, and discussion with the community throughout each iterative cycle. Stages included development of the storyboard, the translation process (forward translation and backtranslation), prelaunch community review, launch and initial community evaluation, and finally, wider launch and evaluation at a viral hepatitis conference.

Results: We produced an app called “Hep B Story” for use with iPad, iPhone, Android tablets, and mobile phones or personal computers. The app is culturally appropriate, audiovisual, interactive, and users can choose either English or Yolŋu Matha (the most common language in East Arnhem Land) as their preferred language. The initial evaluation demonstrated a statistically significant improvement in Hep B-related knowledge for 2 of 3 questions (P=.01 and .02, respectively) and overwhelmingly positive opinion regarding acceptability and ease of use (median rating of 5, on a 5-point Likert-type scale when users were asked if they would recommend the app to others).

Conclusions: We describe the process of development of a bilingual hepatitis B-specific app for Indigenous Australians, using a PAR framework. The approach was found to be successful with positive evaluations.

JMIR Res Protoc 2015;4(2):e70




Chronic hepatitis B (CHB) is endemic in the Indigenous communities of the Northern Territory (NT) of Australia with prevalence rates estimated to be between 3% and 14.2% [1-7], compared with 1% in Australia as a whole [8]. Despite the availability of effective, government subsidized treatments, only 25% of all individuals living with CHB in Australia are estimated to be receiving guideline-based care, with only 5% receiving antiviral therapy [9]. This disparity in rates of hepatitis B and low uptake of treatment is also seen in other Indigenous populations across the world [10,11].

The barriers to Indigenous Australians accessing care for CHB are multifactorial but mainly include the following: gaps in knowledge, low health literacy, ineffective cross-cultural communication, and logistical challenges in accessing the appropriate care. The challenges of effective cross-cultural communication in the context of remote Indigenous communities of the NT have been documented extensively [12-15] with miscommunication said to be pervasive and language translation felt to be only part of the problem. Although many health promotion or information resources exist for hepatitis B [16], the Australian National Hepatitis B strategy [17] and a number of other studies [18-22] specifically highlight the lack of culturally appropriate resources available to facilitate shared understandings of hepatitis B for Indigenous Australians.

We have previously described the results of a qualitative study exploring the knowledge, perceptions, and experiences of remote dwelling Indigenous adults and their health care providers relating to hepatitis B infection [23]. User preferences from this study included the preference for an electronic format with a predominance of pictures; sufficient medical details; human-like figures, not animal analogies; to be in Yolŋu Matha (local Indigenous language) as well as in English; to be interactive; and to use a culturally appropriate world view, building on existing knowledge to facilitate shared understandings. An unrelated scoping study [21] that examined ways to improve and support health education and language interpreting in Aboriginal communities in East Arnhem Land (NT) concluded that user-friendly, interactive, tactile, and aesthetically appropriate resources were most successful at facilitating communication. Involvement of the community in the development of resources in a “bottom-up fashion” was also highlighted as crucial to their eventual success. Multimedia resources were felt to be the most useful with a “touch pad body electronic device” being proposed as a tool to facilitate communication around health issues [24].

In Australia, 64.6% of the population owns a mobile phone [25] with the ability to download electronic apps. There has been an explosion in the development and use of apps with 46 billion downloads worldwide in 2012. This figure is estimated to exceed 200 billion/year by 2017 [26], at which time it is predicted that 50% of mobile phone users will have downloaded a health-related app. The potential to harness this technology as a means to improve health literacy, communication, and treatment uptake is yet to be fully realized. Only recently have published articles started to emerge with respect to the evidence base used to develop health-related apps and any subsequent evaluation of their utility and impact [27,28]. Further, only limited literature exists in this field and it raises concerns regarding the accuracy of information [29,30], alignment of advice with evidence-based guidelines [31], and lack of input from users/patients into product design and evaluation of effectiveness [32]. The data with regard to apps specifically targeted at Indigenous or culturally and linguistically diverse groups are even sparser. We are aware of the development of a number of mental health apps and a rheumatic heart disease app (Menzies School of Health Research, Darwin, Australia) specifically for Indigenous Australians but not of any published literature with respect to the development process of health apps for Indigenous populations.

Participatory Action Research Methodology

Participatory action research (PAR), a cyclical process of reflection, evaluation, and action, where respect for and involvement of the community in all aspects of the research process is an integral part of the methodology, is increasingly recognized as valuable in Indigenous health research [33-35]. There are a number of studies reporting successful outcomes of PAR projects in the context of developing health resources in Indigenous communities [36,37]. This paper aims to describe the process of the development and report the results of the initial evaluation of a culturally appropriate bilingual app about hepatitis B as part of a PAR project.


This project was undertaken in northern Australia between September 2012 and October 2014. It was based at the health clinic of a remote community in Arnhem Land, 521 km northeast of Darwin (the capital city of Australia’s NT). This community has a population of 2124 with an average age of 24 years, of which 88.98% (1890) are Indigenous Australians and only 9.5% (202) of the population speak English as their first language. On average, there are 4.2 people for each available bedroom and 83.19% (1767) of people live in households considered to be overcrowded. The community has 3 general stores, a school, a library, a police station, and a community church.

Herein we report the results of the second phase of this PAR project. Phase 1 (previously reported in detail [23]) was a qualitative study consisting of semistructured interviews carried out with 3 groups of people, namely, key informants (health clinic staff, community health educators, and liver clinic staff from both urban and remote areas, including doctors and nurses from both Indigenous and nonindigenous groups), Indigenous people living with CHB, and Indigenous community members. Interviews explored the following: background of the individuals, their hepatitis B knowledge, their experience of health communication/education about hepatitis B, available resources, and their perspectives about potentially useful educational tools. The results of phase 1 of the study then formed the evidence base for the development of the bilingual app (phase 2). The original impetus for the project came from the staff of the community clinic. Their enthusiasm for the project led to the development of a collaborative research partnership between the Miwatj Health Aboriginal Corporation Community Clinic (an Aboriginal-controlled health service representing communities across East Arnhem Land), the Royal Darwin Hospital Liver Clinic, and the Menzies School of Health Research.

Ethical approval for the study was obtained from the Human Research Ethics Committee of the NT Department of Health and Menzies School of Health Research as well as Miwatj Health Aboriginal Corporation. Figure 1 details the timelines for the development process.

Figure 1. Major stages and timelines for the development of the Hep B Story electronic app. Curved arrows represent time points where episodes of community consultation occurred.
View this figure

Development of the Storyboard

Using the specifications and concepts derived from phase 1 of the PAR process (Figure 2), the project team, which included JD, SB, JSD, LC, SS, and VJ, developed an initial storyboard detailing important ideas, images, and themes to be included in the education tool (Figure 3).

This was then developed by Dreamedia, a Darwin-based graphic design and software company, into an initial screen-by-screen storyboard. Subsequently detailed screen-by-screen scripts were developed by the project team, initially in simple English. These preliminary versions of the app were presented back to the community to facilitate and enable evaluation by clinic staff and liver clinic patients. Appropriate modifications were made according to community input. There were in excess of 20 iterations of the storyboard over the period from February 2013 to July 2014.

Figure 2. Main concepts and ideas taken from phase 1 [23] of the participatory action research (PAR) process to provide the initial evidence base for the culturally appropriate hepatitis B electronic app. The PAR cycle shows the principle stages involved in a PAR project.
View this figure
Figure 3. Screen-by-screen examples of the development process showing first and last versions.
View this figure

Translation Process

The specific language that the app is translated into is “Djambarrpuyngu,” a member of the “Yolŋu Matha” group of Australian Indigenous languages used by Yolŋu people in Northeast Arnhem Land. The term “Yolŋu Matha” will be used in this paper to refer to the language from this point forward as this is the term used in the app in line with community wishes. Two experienced interpreters from the local community were identified, one allocated to do the forward translation (ie, English to Yolŋu Matha) and one to undertake the backtranslation (ie, Yolŋu Matha back to English, so as to check the accuracy of the translation) according to the World Health Organization guidelines [38]. Both interpreters had previously translated health education materials and participated in a pretranslation hepatitis B education session to enable familiarization with the material. Conceptual equivalence (aiming for shared understanding of a word or phrase rather than a word-for-word literal translation) was discussed and encouraged wherever appropriate. Backtranslation was undertaken independently by the second of the interpreters, then the English checked, and clarified by JD (English-speaking doctor with experience in a cross-cultural environment) and SB (bilingual Aboriginal Health Worker), again with emphasis on conceptual and cultural equivalence rather than linguistic equivalence. The final Yolŋu Matha translation was reviewed again by SB. Voice-overs were recorded at Dreamedia studios in both English and Yolŋu Matha.

Prelaunch Community Review

Functional prototype versions of the electronic app were produced on 4 occasions by Dreamedia and presented to the community for review. Discussion and input from clinic staff (both Indigenous and nonindigenous) and Indigenous liver clinic patients were sought on each occasion. Changes were then incorporated into the next version of the app and the process was repeated until unanimous approval was achieved.

Launch and Initial Community Evaluation

In July 2014, a launch event was organized in the community, which involved presentation of the app by JD and SB, and an invitation to take part in the initial evaluation process. The evaluation questionnaires had sections to be completed before and after exploring the app. People were guided through the questionnaire process in real time by a bilingual research assistant who translated the questions and answers into Yolŋu Matha where needed or requested.

Viral Hepatitis Conference Launch and Initial Evaluation

The inaugural Indigenous Peoples’ Conference on Viral Hepatitis in Alice Springs, NT (September 2014) was chosen as an appropriate place to launch the app to the wider sector. The app was presented as part of an exhibit where individuals could explore the app in their own time. We invited all conference delegates to help evaluate the app using the aforementioned questionnaire.

Evaluation Questionnaire Analysis

Data were entered into Microsoft Excel 2010 (Microsoft, Redmond, WA, USA) and analyzed using Stata version 13 (StataCorp, College Station, TX, USA). Overall preapp and postapp knowledge scores were created by calculating a total score out of 6 (based on the number of correct responses to Q1 and Q4) and presented as percentages. Quantitative continuous variables were presented as mean ± standard deviation for normally distributed parameters and median ± interquartile range (IQR) for non-normally distributed parameters. Bivariate analyses were performed using Student t test, paired Student t test for preapp and postapp knowledge comparisons, Fisher exact test for comparisons with any cell value less than 5, and Mann-Whitney test for nonparametric data.

Likert-type items (Q2 and Q3) were treated as ordinal data, presented as median values with frequencies, and associations between groups were calculated using Kendall Tau-b [39]. For each Likert type, item 1 equates to “strongly agree, 2 “agree,” 3 “neutral,” 4 “disagree,” and 5 “strongly disagree.” For positive statements, allocated scores were the inverse of the assigned number to enable higher scores to reflect more positive responses.

Main Findings

The app we produced is called “Hep B Story” and is available as a free download for Apple devices through the Apple App Store, for Android devices through the Google Play Store, and as a Web-based app through the Menzies School of Health Research website. The app’s title screen allows the user to choose the language as either English or Yolŋu Matha and they can navigate from the beginning to the end through the entire app (except for the “women’s business” section), or choose to go to the “chapter select” screen to skip to a specific section or to enter the women’s business section. There are 7 “chapters” as detailed in Figure 4, some of which contain animations. The treatment section includes a game, which involves dragging tablets into a man’s mouth each day. If you do not get him to take the medicine consistently enough, his hepatitis B virus becomes resistant to the tablets and his liver becomes diseased. Each screen has an audio button, which will play the voice-over in English or Yolŋu Matha and the text is also displayed on the screen in the preferred language.

Figure 4. The contents of the Hep B Story app.
View this figure

Development of the Storyboard

As with the text, the images for each screen were discussed in detail over several iterations. Particular emphasis was placed on getting an appropriate balance of gender representation throughout the app, as well as a desire for the individual on the front screen to be gender neutral. Initially, the title screen figure had brown skin, but after community review it was felt that this may imply that all people with hepatitis B have brown skin, and therefore, the color was changed to blue to be ethnically neutral.

The cultural appropriateness of an image to represent sexual transmission was the subject of much discussion. This mode of transmission was, however, downplayed at the community’s request for reasons of stigma and also as the main route of transmission in this population is thought to be mother-to-child transmission during parturition.

The screen representing liver function initially had visual representations of various liver functions (eg, fruit to represent its role in helping to process food). On consultation with the community, this was felt to be a culturally inappropriate use of lateral thinking and was described as a decoy and confusing. People preferred to remove all the pictures and just have the words over the liver images as in the final version (Figure 3).

A separate women’s business section was a recurrent request. It was important to community members that this was not something that you would “stumble upon” while looking through the app and it had to be separate, warranting an active decision to visit this section. “Women’s business” is the commonly used local term for health and other matters specific to women, such as pregnancy.

Translation Process

The script was translated taking into account context and cultural appropriateness. The Yolŋu Matha version was much longer than the English version (10 pages of text as opposed to 5). There were a number of reasons for this: Yolŋu Matha is a more verbose storytelling language; there is not a direct translation for many terms, for example, “Hep B” in the backtranslation was “those invisible germs that are the sickness in your blood called Hep B.” For clarity, this phrase was then used on each occasion Hep B was mentioned, and therefore increased the number of words. The text was translated meticulously by a senior male Yolŋu elderly person with great attention to detail, contextualization of the text, and cultural appropriateness. For example, the sentence talking about how one of the liver’s functions is to produce clotting factors in the translated version says “it is like a factory that makes good oil, it will help the blood to clot and fight sickness.” A senior female elderly person backtranslated this again with the same attention to detail. Issues raised following the backtranslation were literal interpretation of concepts, the difficulty when there are many words for one thing, and the importance of choosing the most appropriate one for the specific context needed.

Prelaunch Community Review

One of the key requests from phase 1 of this study was for a visual tool; however, an app prototype without the full text appearing alongside the audio was not well received. The community consensus was to have the full text there so that people could either listen or read depending on their preference.

A request was made to add kava drinking (a drink made from the extract of the root Piper methysticum with sedative and anesthetic properties, which is commonly used as a recreational drug in parts of Arnhem Land) to the screen about the dangers of the combination of alcohol and hepatitis B for the liver. An image of a family with several members infected was also suggested to be included on the screen talking about epidemiology.

Launch and Initial Community Evaluation

The launch in the community was met with excitement and pride at seeing an electronic app in Yolŋu Matha. The evaluation questionnaire was completed by 16 people, median age 34 years (IQR 30-59), with 12 (75%) being women. Results are presented in Tables 1 and 2.

Viral Hepatitis Conference Launch and Initial Evaluation

The launch and 3-day exhibit at the Viral Hepatitis Conference in Alice Springs, NT (September 2014) were also met with a positive response. The questionnaire was completed by 56 people, median age 45 years (IQR 36-54), with 50 (89%) being women. Results are presented in Tables 1 and 2.

Table 1. Demographics of participants and results of the opinion-based component of the evaluation questionnaire.

Community launch group
Conference delegate group
P valuea
Demographics of groups

Age in years median (interquartile range)34 (30-59)45 (36-54).34

Indigenous status (%)94 (15/16)21 (12/56)<.001

Female gender (%)75 (12/16)89 (50/56).20
Self-rated hepatitis B knowledge (% who strongly agree or agree)a

Never heard of hepatitis B50 (8/16)4 (2/56)<.001

I knew a lot about hepatitis B31 (5/16)66 (37/56).02
Postapp opinion using the 5-point Likert scale Median score, % giving that answer (n/N)b

I found the app easy to use (for Q3a)5, 67 (10/15)5, 80 (44/55).29

Easy to understand (for Q3b)5, 67 (10/15)5, 75 (41/55).43

Contained enough information for my needs (for Q3c)5, 47 (7/15)5, 67(37/55).42

Contained too much information for my needs (for Q3d)2, 50 (6/12)4, 17 (9/54).005

I would recommend the app to my family and friends (for Q3f)5, 69 (9/13)5, 67 (36/54).72

Use the app again myself (for Q3g)5, 64 (9/14)5, 55 (29/53).56

aP values are comparisons between the community launch group and the conference delegate group.

b“Self-rated hepatitis B knowledge” and “Postapp opinion” constituted the results of opinion-based questions.

Table 2. Results of the knowledge-based component of the evaluation questionnaire.
Knowledge-based questionsCommunity launch group (n=16)aConference delegate group (n=56)a

PreappPostappP valuebPreappPostappP valueb
Can you name 3 ways by which hepatitis B can be passed from one person to another?33 (16-50)58 (39-77).01100 (100-100)97 (96-100).32
If you have hepatitis B what is the best way to tell if the virus is causing damage to your liver?25 (1-49)38 (11-64).1692 (85-99)89 (83-99).56
If you have hepatitis B what can you do to help your liver to stay healthy (name 2 things)?47 (29-65)50 (28-72).5884 (76-93)94 (88-99).02

aValues are presented as mean score % (95% CI).

bP value for paired t test comparing preapp versus postapp knowledge scores.

Evaluation Questionnaire Results

Overall, 72 individuals completed the evaluation questionnaire of whom 62 (86%) were women and 27 (38%) were Indigenous Australians, with a median age of 44 years (IQR 34-54). There was a good representation of individuals from most parts of Australia with only Tasmania and Australian Capital Territory not being represented. With regard to preapp and postapp knowledge assessment, there was a statistically significant increase in the first knowledge-based question in the community launch group (P=.01) and in the final one in the conference delegate group (P=.02, Table 2).

Participants’ opinions of the app after use were generally positive, with 5 of the 6 questions on this achieving a median rating of 5 on the 5-point Likert-type scale. The free text comments were overwhelmingly positive with multiple references to ease of use, culturally appropriate graphics, and the importance of being able to both read and listen in Yolŋu Matha. Multiple requests were made for the app to be translated into other languages. A recurrent criticism was made referencing the lack of inclusivity with regard to gender and sexually diverse communities by the wider key informant group.

Principal Findings

We describe in detail the process of development of the “Hep B Story” app, a culturally appropriate educational resource about CHB, through a community partnership using a PAR framework. “Hep B Story” is the first app to be produced in Djambarrpuyngu, a member of the Yolŋu Matha group of Australian Indigenous languages spoken widely in the North East Arnhem Land in Australia’s NT. The continuous iterative cycle of consultation, evaluation, and adaptation has led to the production of a tool that the community was proud of and excited about. Initial evaluations from both the community and a wider group of key stakeholders have been overwhelmingly positive.

The production of apps and literature concerning their development are rapidly increasing; however, for the vast majority of health-related apps, there is currently no standardized development or regulatory process for assessment of their quality or effectiveness [27]. There are few published descriptions of the process of health app development and it is often difficult to ascertain, when considering using or recommending an app, who exactly has produced the content and if end users have been involved in the process. Recent reviews of apps for specific disease areas have highlighted concerns regarding factual accuracy, lack of end user involvement, and the effectiveness of apps to add value to standard health care practice [28]. A Cochrane review [31] looking specifically at apps facilitating the self-management of asthma (>100 apps available) concluded that the current evidence base (only 2 studies included) is not sufficient to advise clinical practitioners, policy makers, or the general public regarding app effectiveness. A number of reviews on dermatology apps directed toward skin cancer screening have reported wide ranges of sensitivity and specificity for the diagnosis of melanoma [30,40], with one reporting that 88.2% of biopsy-proven melanoma was classified by the app as “medium risk” and individuals were thus advised to “monitor only” [41]. A trial protocol has been published for an evaluation of the effectiveness of a suicide-prevention app in Indigenous Australian youth (currently recruiting); however, no details are currently available as to how the app was developed [42]. Recently, regulations have been introduced by both the United States Food and Drug Administration and the European Union for “medical device” apps (those intended as an accessory to a regulated medical device or those that transform a mobile platform into a regulated medical device, such as an app intended to diagnose cardiac arrhythmias). These regulations do not currently apply to health information or education apps [43]. A number of app clearinghouse websites are now available with varying levels of review and accreditation of health-related apps [27].

In the context of Indigenous health, community partnerships and the use of PAR methodologies have been shown to help break down barriers to communication [34,36], and understandably people respond to information in their own language more positively than that in a second language. Our experience highlights the importance of meticulous translation, backtranslation, and the attention to detail needed to ensure that the messages you are giving are both linguistically and contextually correct and will be understood in the way intended. There is great potential for harm if this process is not robust. This was highlighted in phase 1 of the PAR process when during the process of the interviews it became apparent that there was a lack of shared understanding of the word “silent” between nonindigenous health workers and Indigenous patients in the context of hepatitis B. The health workers used this to describe the asymptomatic nature of hepatitis B noted at times, whereas the patients understood this to mean that the sickness is brought about by sorcery [23].

We also highlight the great value that can be gained from repeated reviews of and conversations about a product throughout the development process, particularly having a real prototype version of the app to comment on. This allowed multiple changes and additions to the visual appearance and aesthetics of the app such as the color of the person and the images on the liver function screen, which would have been difficult to tease out from verbal-only communication. It also provides multiple opportunities to open up communication facilitating collective agreement making and allowing “bottom-up” changes to occur, which have been highlighted as crucial to change in the context of remote Indigenous communities [24].

Initial evaluations of the Hep B Story app were overwhelmingly positive, with all but 1 question (question 3d) assessing user’s opinions achieving a median rating of 5 on a 5-point Likert-type scale. Question 3d was a negative statement compared with all the others, which were positive and it may be that this was confusing or difficult to translate as the lowest score for this question came from the community evaluation. This needs further clarification and exploration before the design of any subsequent evaluation. It will also be important to consider why the numbers of people completing the questionnaire in the community launch group were much lower than the conference delegate group. It may be that due to different cultural and communication protocols, as a questionnaire-based evaluation is not the optimal methodology to use in this setting.

There was a significant increase in knowledge after use of the app for questions 1a in the community launch group (P=.01) and 1c in the conference delegate group (P=.02). Although the mean score increased from 25% to 38% in the community launch group for knowledge question 1b, this was not statistically significant (P=.16), and it is possible that this would reach significance with a larger sample size. It is also important to acknowledge the persistence of low levels of knowledge in the community launch group even after exploring the app and pertinent to critically examine whether this is likely to be a problem with the app itself or the way the questions were asked and the methodology used in this initial evaluation. Culturally appropriate measurement of the impact of interventions aimed at improving health knowledge is problematic, with a tendency to attempt to quantify “knowledge” without having any validated tools to achieve this in an Indigenous setting. There are a number of studies in progress in an Australian Indigenous setting using adapted versions of validated health literacy measurement tools such as the Health Literacy Management Scale and Health Literacy Questionnaire, which will hopefully provide much needed information in this area [44-46]. Christie and Verran [24] have suggested that, in the context of East Arnhem Land, low health literacy is not so much a knowledge problem but a need for allowing shared understandings to develop, and our work from the first phase of this PAR project would generally concur with this [23]. It is, therefore, worthwhile considering the cultural appropriateness of any kind of “knowledge measurement” within our further evaluation. It is important that a larger more formal evaluation of this app does take place to confirm and add more detail regarding the impact on knowledge and ultimately behavior change; however, it may be that an interview-based evaluation methodology is more appropriate. Because of the nature of the PAR process, it would be difficult to objectively evaluate this app’s effectiveness and acceptability in a community that has already been so involved in its development. We are therefore planning a separate evaluation in a different location where Yolŋu Matha is spoken and hepatitis B is common.

Even in a community where there has been significant engagement, involvement, and interest in the hepatitis B project, levels of knowledge around CHB were low with 50% (8/16) of people having never heard of it. This is consistent with the work done in the Torres Strait region of Australia [19,22], which reported a lack of awareness and/or knowledge of CHB and the measures to reduce its health impact both at the patient and at the health care provider level. The free text comments from the conference delegate group with respect to gender and sexually diverse communities highlighted the conflicts that can arise in PAR projects. Obtaining the right balance between cultural appropriateness and community wishes versus inclusivity of potentially underrepresented groups in this context can be challenging.

Limitations to extensive engagement and involvement of end users in the process of app development include the length of time needed to undertake this process robustly, particularly in communities where English is not the first language, and worldviews of health are very different. This is especially relevant to digital technologies where the pace of change is so fast. As an intended consequence of the methods used to produce the app, it is very specifically tailored to the culture and needs of Yolŋu people; therefore, its translatability to other groups is always going to require further consultation and adaptation. Although this process can be streamlined when the framework and technical coding for the app are available, it is still a significant undertaking. Another obvious issue is that not everyone has access to a mobile phone or tablet device. As such, we felt it crucial to also have the app available through an Internet site that can be accessed from nonportable computers such as those in health care facilities.

The potential for adaptation to other languages is important in the context of hepatitis B, which disproportionately affects Indigenous people both in Australia and across the world. There is also great potential to personalize and incorporate clinical tools into the app such as tracking of blood test results, triggers for follow-up, medication reminders, and even self-assessment tools. One example would be adapting the “number connection test,” which is a widely used method for detecting the reduced spatial awareness and coordination present in hepatic encephalopathy. This currently consists of a timed test connecting numbered dots on a sheet of paper, which could be adapted into a game-style version and included in the app. The section about alcohol and kava could incorporate harm-reduction strategies, monitoring of consumption, and abstinence encouragement tools.


Health-related apps have a huge potential to contribute and impact positively on health care; however, there is also substantial risk of harm in the absence of an evidence base to guide standards, regulation, and development. This is particularly true for populations where health beliefs and worldview are different and English is not the first language. These are the very populations that are most affected by CHB. We described using a PAR framework with both end users and key informants providing their inputs into the content and development of a hepatitis B-specific app. Although this process was time consuming, the approach was very successful, with a majority of participants providing positive responses. The long-term effectiveness with respect to improving patient’s health literacy leading to behavior change and increased treatment uptake will be evaluated over time.


We thank the translation and Yolŋu Matha voice-over team, Maratja Dhamarrandji and Dorothy Gapany; the English voice-over team, Tristan Etherington and Christine Mills; the design and development team, Chris Torralba, Krupa Patel, Aoife Dowling, and Alice Plate as well as all study participants and local health clinic staff who all so generously gave their thoughts and time to contribute to this study. This study was also supported in part by a National Health and Medical Research Council project grant (Grant No GNT 1060811).

Conflicts of Interest

JD and JSD have received an unrestricted educational grant from Gilead Sciences to finance this project with the exception of their time, which is supported by the National Health and Medical Research Council (PhD scholarship to JD, early career fellowship to JSD) and Sidney Myer funds (top-up scholarship to JD).

  1. Barrett EJ. Hepatitis B in Australian Aborigines and Torres Strait Islanders: Geographical, age and familial distribution of antigen subtypes and antibody. Aust N Z J Med 1976 Apr;6(2):106-111. [Medline]
  2. Gardner ID, Wan X, Simms PA, Worswick DA, Burrell CJ, Mathews JD. Hepatitis B virus markers in children and staff in Northern Territory schools. Med J Aust 1992 May 04;156(9):638-641. [Medline]
  3. Wood N, Warlow M, Quinn H, Selvey C, Lum G, McIntyre P, et al. Establishment of a surveillance system (utilising Midwifes Data Collection Systems) for monitoring the impact of hepatitis B vaccination on the population prevalence of chronic hepatitis B virus infection in Australia. Aust N Z J Public Health 2008 Jun;32(3):272-275. [CrossRef] [Medline]
  4. Schultz R, Romanes F, Krause V. Hepatitis B prevalence and prevention: Antenatal screening and protection of infants at risk in the Northern Territory. Aust N Z J Public Health 2008 Dec;32(6):575-576. [CrossRef] [Medline]
  5. Carroll E, Page W, Davis JS. Screening for hepatitis B in East Arnhem Land: A high prevalence of chronic infection despite incomplete screening. Intern Med J 2010 Nov;40(11):784-787. [Medline]
  6. Dent E, Selvey CE, Bell A, Davis J, McDonald MI. Incomplete protection against hepatitis B among remote Aboriginal adolescents despite full vaccination in infancy. Commun Dis Intell Q Rep 2010 Dec;34(4):435-439. [Medline]
  7. Liu B, Guthridge S, Li SQ, Markey P, Krause V, McIntyre P, et al. The end of the Australia antigen? An ecological study of the impact of universal newborn hepatitis B vaccination two decades on. Vaccine 2012 Nov 26;30(50):7309-7314. [CrossRef] [Medline]
  8. MacLachlan JH, Allard N, Towell V, Cowie BC. The burden of chronic hepatitis B virus infection in Australia, 2011. Aust N Z J Public Health 2013 Oct;37(5):416-422. [CrossRef] [Medline]
  9. Allard N, MacLachlan JH, Cowie B. P680 A national health system response to chronic hepatitis B: Using population data to define gaps in clinical care provision. J Hepatol 2014;60(1):S295-S296. [CrossRef]
  10. Osiowy C, Simons BC, Rempel JD. Distribution of viral hepatitis in indigenous populations of North America and the circumpolar Arctic. Antivir Ther 2013;18(3 Pt B):467-473. [CrossRef] [Medline]
  11. Robinson T, Bullen C, Humphries W, Hornell J, Moyes C. The New Zealand Hepatitis B Screening Programme: Screening coverage and prevalence of chronic hepatitis B infection. N Z Med J 2005 Mar 11;118(1211):U1345. [Medline]
  12. Cass A, Lowell A, Christie M, Snelling PL, Flack M, Marrnganyin B, et al. Sharing the true stories: Improving communication between Aboriginal patients and healthcare workers. Med J Aust 2002 May 20;176(10):466-470. [Medline]
  13. Trudgen R. Why Warriors Lie Down and Die: Towards an Understanding of Why the Aboriginal People of Arnhem Land Face the Greatest Crisis in Health and Education Since European Contact: Djambatj Mala. Darwin, NT, Australia: Aboriginal Resource and Development Services; 2000.
  14. Vass A, Mitchell A, Dhurrkay Y. Health literacy and Australian Indigenous peoples: An analysis of the role of language and worldview. Health Promot J Austr 2011 Apr;22(1):33-37. [Medline]
  15. Lowell A, Maypilama E, Yikaniwuy S, Rrapa E, Williams R, Dunn S. "Hiding the story": Indigenous consumer concerns about communication related to chronic disease in one remote region of Australia. Int J Speech Lang Pathol 2012 Jun;14(3):200-208. [CrossRef] [Medline]
  16. HepBHelp. HepBHelp: Who We Are.   URL: [accessed 2015-01-08] [WebCite Cache]
  17. Australian Government, Department of Health and Ageing. National Hepatitis B Strategy 2010–2013. Canberra, Australia: Australian Government, Department of Health and Ageing; 2010.   URL: http:/​/www.​​internet/​main/​publishing.nsf/​Content/​6433B4DF24C1E803CA257BF0001E740B/​$File/​hepb.​pdf [accessed 2015-06-03] [WebCite Cache]
  18. Olsen A, Wallace J, Maher L. Responding to Australia's National Hepatitis B Strategy 2010-13: Gaps in knowledge and practice in relation to Indigenous Australians. Aust J Prim Health 2014;20(2):134-142. [CrossRef] [Medline]
  19. Preston-Thomas A, Fagan P, Nakata Y, Anderson E. Chronic hepatitis B: Care delivery and patient knowledge in the Torres Strait region of Australia. Aust Fam Physician 2013 Apr;42(4):225-231 [FREE Full text] [Medline]
  20. Christie M. The Chronicle. 2015. Re-Thinking: Health Literacy in Remote Communities   URL: [accessed 2015-01-08] [WebCite Cache]
  21. Charles Darwin University. East Arnhemland Client Health Education and Health Interpreting Scoping Report and Project Proposal. Canberra, Australia: Office of Aboriginal and Torres Strait Islander Health; 2010 Jun.   URL: [accessed 2015-01-08] [WebCite Cache]
  22. Wallace J, Pitts M, McNally S, Ward J, Nakata Y, Fagan P. A Situational Analysis of Chronic Hepatitis B in the Torres Strait. LaTrobe University, Melbourne: Australian Research Centre in Sex, Health and Society; 2011.   URL: http:/​/www.​​profile/​Jack_Wallace/​publication/​258344792_A_Situational_Analysis_of_Chronic_Hepatitis_B_in_the_Torres_Strait/​links/​0c960528008975bbc3000000.​pdf [accessed 2015-06-03] [WebCite Cache]
  23. Davies J, Bukulatjpi S, Sharma S, Davis J, Johnston V. "Only your blood can tell the story": A qualitative research study using semi-structured interviews to explore the hepatitis B related knowledge, perceptions and experiences of remote dwelling Indigenous Australians and their health care providers in northern Australia. BMC Public Health 2014;14:1233 [FREE Full text] [CrossRef] [Medline]
  24. Christie M, Verran H. The touch pad body: A generative transcultural digital device interrupting received ideas and practices in Aboriginal health. Societies 2014 May 27;4(2):256-264. [CrossRef]
  25. Godfrey M. Sydney Morning Herald. 2013 Jul 29. Two-thirds of Aussies own a smartphone: Report   URL: http:/​/www.​​digital-life/​mobiles/​twothirds-of-aussies-own-a-smartphone-report-20130729-2quj1.​html [accessed 2015-01-08] [WebCite Cache]
  26. Portio Research. Mobile Applications Futures 2013-2017. Wilts, UK: Portio Research; 2013.   URL: http:/​/www.​​en/​mobile-industry-reports/​mobile-industry-research-reports/​mobile-applications-futures-2013-2017.​aspx [accessed 2015-01-08] [WebCite Cache]
  27. Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: Strategies for healthcare providers and healthcare organizations. Transl Behav Med 2014 Dec;4(4):363-371 [FREE Full text] [CrossRef] [Medline]
  28. Mosa AS, Yoo I, Sheets L. A systematic review of healthcare applications for smartphones. BMC Med Inform Decis Mak 2012 Jul 10;12:67 [FREE Full text] [CrossRef] [Medline]
  29. Bender JL, Yue RY, To MJ, Deacken L, Jadad AR. A lot of action, but not in the right direction: Systematic review and content analysis of smartphone applications for the prevention, detection, and management of cancer. J Med Internet Res 2013 Dec 23;15(12):e287 [FREE Full text] [CrossRef] [Medline]
  30. Brewer AC, Endly DC, Henley J, Amir M, Sampson BP, Moreau JF, et al. Mobile applications in dermatology. JAMA Dermatol 2013 Nov;149(11):1300-1304. [CrossRef] [Medline]
  31. Marcano Belisario JS, Huckvale K, Greenfield G, Car J, Gunn LH. Smartphone and tablet self management apps for asthma. Cochrane Database Syst Rev 2013 Nov 27;11:CD010013. [CrossRef] [Medline]
  32. Wallace LS, Dhingra LK. A systematic review of smartphone applications for chronic pain available for download in the United States. J Opioid Manag 2014 Feb;10(1):63-68. [CrossRef] [Medline]
  33. Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health 2006 Oct;60(10):854-857 [FREE Full text] [CrossRef] [Medline]
  34. Esler DM. Participatory action research in indigenous health. Aust Fam Physician 2008 Jun;37(6):457-459 [FREE Full text] [Medline]
  35. Pyett P. Working together to reduce health inequalities: Reflections on a collaborative participatory approach to health research. Aust N Z J Public Health 2002 Aug;26(4):332-336. [Medline]
  36. Mendenhall TJ, Berge JM, Harper P, GreenCrow B, LittleWalker N, WhiteEagle S, et al. The Family Education Diabetes Series (FEDS): Community-based participatory research with a midwestern American Indian community. Nurs Inq 2010 Dec;17(4):359-372. [CrossRef] [Medline]
  37. Schoen D, Balchin D, Thompson S. Health promotion resources for Aboriginal people: Lessons learned from consultation and evaluation of diabetes foot care resources. Health Promot J Austr 2010 Apr;21(1):64-69. [Medline]
  38. World Health Organization. Process of Translation and Adaptation of Instruments. Geneva, Switzerland: World Health Organization; 2015.   URL: [accessed 2015-01-08] [WebCite Cache]
  39. Boone HN, Boone DA. Analyzing Likert data. J Ext 2012 Apr;50(2):1-5 [FREE Full text]
  40. Wolf JA, Moreau JF, Akilov O, Patton T, English JC, Ho J, et al. Diagnostic inaccuracy of smartphone applications for melanoma detection. JAMA Dermatol 2013 Apr;149(4):422-426 [FREE Full text] [CrossRef] [Medline]
  41. Ferrero NA, Morrell DS, Burkhart CN. Skin scan: A demonstration of the need for FDA regulation of medical apps on iPhone. J Am Acad Dermatol 2013 Mar;68(3):515-516. [CrossRef] [Medline]
  42. Shand FL, Ridani R, Tighe J, Christensen H. The effectiveness of a suicide prevention app for indigenous Australian youths: Study protocol for a randomized controlled trial. Trials 2013;14:396 [FREE Full text] [CrossRef] [Medline]
  43. Charani E, Castro-Sánchez E, Moore LS, Holmes A. Do smartphone applications in healthcare require a governance and legal framework? It depends on the application!. BMC Med 2014 Feb 14;12:29 [FREE Full text] [CrossRef] [Medline]
  44. Crengle S, Smylie J, Kelaher M, Lambert M, Reid S, Luke J, et al. Cardiovascular disease medication health literacy among Indigenous peoples: Design and protocol of an intervention trial in Indigenous primary care services. BMC Public Health 2014;14:714 [FREE Full text] [CrossRef] [Medline]
  45. Parker EJ, Misan G, Chong A, Mills H, Roberts-Thomson K, Horowitz AM, et al. An oral health literacy intervention for Indigenous adults in a rural setting in Australia. BMC Public Health 2012;12:461 [FREE Full text] [CrossRef] [Medline]
  46. Batterham RW, Buchbinder R, Beauchamp A, Dodson S, Elsworth GR, Osborne RH. The OPtimising HEalth LIterAcy (Ophelia) process: Study protocol for using health literacy profiling and community engagement to create and implement health reform. BMC Public Health 2014 Jul 07;14:694 [FREE Full text] [CrossRef] [Medline]

CHB: chronic hepatitis B
IQR: interquartile range
NT: Northern Territory
PAR: participatory action research

Edited by G Eysenbach; submitted 08.01.15; peer-reviewed by A Lowell, C Treloar, D Victorson; comments to author 07.04.15; revised version received 14.04.15; accepted 16.04.15; published 10.06.15


©Jane Davies, Sarah Bukulatjpi, Suresh Sharma, Luci Caldwell, Vanessa Johnston, Joshua Saul Davis. Originally published in JMIR Research Protocols (, 10.06.2015.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.