This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://resprot.jmir.org, as well as this copyright and license information must be included.
Ethnic minority women (EMW) living in mountainous areas of northern Vietnam have disproportionately high infant and maternal mortality rates as a result of low maternal health knowledge, poverty, and remoteness from low-capacity health centers.
The objective of this study was to describe the protocol for the development and evaluation of the mMom intervention, which is an integrated mobile health (mHealth) system designed to improve maternal and infant health knowledge, and behavior among women in remote areas of Thai Nguyen, Vietnam.
This project featured the following four phases: (1) development of an mHealth platform integrated into the existing health management information system in partnership with the provincial health department; (2) ethnographic fieldwork and intervention content development; (3) intervention piloting and implementation; and (4) evaluation of the intervention’s impact on participants’ maternal health knowledge, behavior, and interactions with the health system.
The mMom project development process resulted in the following: (1) the successful development of the mMom system, including the mHealth platform hardware and integration, the intervention plan and content, and the monitoring and evaluation framework; (2) the piloting and implementation of the intervention as planned; and (3) the implementation of the monitoring and evaluation framework components.
This protocol outlines the development of the mMom intervention and describes critical next steps in understanding the impact of the intervention on participants and the wider health system in Thai Nguyen province, Vietnam.
Vietnam has made remarkable progress in improving maternal, newborn, and child health (MNCH), but ethnic minority women (EMW) living in mountainous and remote areas still lag far behind. Although Vietnam’s overall under-five mortality rate declined from 50.8 per thousand live births in 1990 to 21.7 per thousand in 2015 [
Health outcome disparities among ethnic minorities in northern Vietnam are profound, primarily because of sociostructural factors including poor education, low economic and social status, and rural or remote residence [
Over the past decade, mobile health (mHealth) initiatives have burgeoned in both low- and high-income contexts, supported by the high penetration of inexpensive mobile phone services. mHealth has been widely applied to address the stark global inequities in maternal and infant mortality in low- and middle-income countries, as interconnected economic, political and sociocultural factors have impeded progress on MNCH indicators and mHealth may hold the potential to mitigate some of these barriers.
With regard to mHealth acceptability, several recent studies in diverse regions have identified receptive attitudes from target communities toward mHealth projects, ranging from 70% to 90% in different interventions [
Available literature indicates high acceptability of mHealth among health staff and that mHealth projects can positively impact community health work and systems. mHealth interventions have been found to increase staff efficiency and improve data entry and monitoring [
Although there has been an increased application of mHealth to support MNCH, evidence of the outcomes of SMS-based interventions for MNCH is still limited [
Vietnam’s mobile phone network and market have undergone swift development as the country has experienced rapid economic growth since the late 1980s. Vietnam is categorized as a low-income country, but the number of mobile subscriptions over its population is higher than that of many developed countries. In 2015, there were 130 mobile cellular subscriptions for every 100 people in Vietnam [
Furthermore, although literature on the development of mHealth interventions is growing, certain areas remain understudied. A gap in current literature is the relative absence of studies examining policy and management frameworks to support adoption of mHealth services, with a specific gap related to governance and partnerships with regional and national partners. Issues such as the coordination for mHealth projects among different government bureaus, and protocols for assuring quality of information, require further investigation. This study aimed to address this noted gap through creating multilevel partnerships between commune-, district-, and provincial-level health authorities and documenting the challenges of an integrated, intergovernmental, and interdepartmental project to contribute lessons currently lacking in the literature.
The objective of this paper is to describe the protocol for the development and evaluation of the mMom intervention, which is an integrated mHealth system designed to improve maternal and infant health knowledge and behavior among women in a remote area of Thai Nguyen province in mountainous northern Vietnam.
The Government of Vietnam has demonstrated increasing interest in health system strengthening and mHealth over the past decade, creating a favorable policy environment and suitable context for partnerships toward greater integration of mHealth. In 2009, the Ministry of Health established a Central Health Information and Technology Institute, and telemedicine and Web-based electronic medical records systems were subsequently established to strengthen health management information systems (HMISs). In 2012, the Ministry of Health introduced a new set of administrative directives on medical and health information technology (IT), and the Thai Nguyen Provincial Health Department (TNHD) received funds from Atlantic Philanthropy to computerize its HMIS, supported by the Institute of Population Health and Development (PHAD). In Thai Nguyen province, all hospitals and 181 communes in 9 districts are fully integrated into an electronic record management system, and 100% of commune health centers are connected by high speed cables to district and provincial health centers.
Due to the central and provincial government’s commitments to use IT to improve health, the mMom project fit well within national and local priorities. The mMom software and its operation was developed and managed by Vietnam electronic health (eHealth) Medical Investment and Communication. The project built a new mHealth component, aimed at health communication, as part of the existing HMIS for maternal and child health. However, this component can also function independently and can be applied to other modules of the HMIS. The mMom database was created as an integrated component of the HMIS through using each patient’s unique identity code. The platform aimed to coordinate and add support to commune health workers’ existing tasks in monitoring pregnancy and new motherhood in community women. In addition, the project’s integration into the HMIS aimed to improve data collection and monitoring, with the goal of supporting future MNCH programming evaluation and improvements due to strengthened data collection systems.
Ethnographic fieldwork was conducted at the mMom project outset to identify gaps in current MNCH approaches in the intervention site (Dinh Hoa district, Thai Nguyen, Vietnam) and to inform the mHealth intervention. Fieldwork methodology included focus group discussions (bringing people together to explore their attitudes and experiences on a topic of interest in a group setting); in-depth interviews (guided conversations between a researcher and an individual, which collect specific information on their perceptions and experiences) with the target populations of currently pregnant women, new mothers, their families, and health workers; and document review. Focus groups and interviews were selected as the methods most suited to achieving the fieldwork goals of assessing the extent of mobile phone use in the district, literacy barriers, social and cultural issues affecting phone ownership and use, health service utilization, capacity and response of health workers, organizational factors that may impact mHealth promotion, and technical constraints such as mobile signal coverage.
The intervention was developed based on several well-known conceptual frameworks for mHealth and maternal health. These included the following: (1) the World Health Organization (WHO) framework and standards for country health information systems, which emphasizes reliable and timely health information as an essential foundation of effective public health systems and provides guidelines for strengthening the key components of health information systems [
The intervention consists of 2 SMS message programs designed to support women’s health during pregnancy and new motherhood. The mMom SMS messages were developed using MAMA [
Each SMS program was designed to include the following 4 types of messages: informational and educational, reminder, interactive, and scanning. Informational, educational and reminder messages provided one-way information and reminded women to take critical actions, such as getting a tetanus immunization. Interactive or three-way messages requested women to respond to monitoring questions, such as whether the participant has recently visited the health center. If women do not respond, or if their responses suggest a risk, the database automatically informs that individual participant’s health worker so that they may reach out to the woman. Within each SMS program, approximately 15% (12/75) of all messages were interactive messages. Finally, the program linked the interactive messages to the HMIS database of the TNHD to scan for relevant information and take the next action appropriately. For instance, if the system scans the HMIS and learns that a woman already obtained her polio shot, the system will then skip the reminder message for the polio immunization. Each type of message was sent at the appropriate time for the stage of a woman’s pregnancy; the timing was customized within the program algorithms.
The intervention site was determined through discussion with the Dinh Hoa District Health Center. All communes in Dinh Hoa district were stratified into centrally located, middle and remote communes, and then random selection was applied to choose 8 project communes and 4 control communes in the district.
The mMom intervention target population included EMW living in Dinh Hoa district who had a current pregnancy or who had a child aged less than 1 year. The sampling frame within the project communes included all pregnant women, new mothers, and women of childbearing age who were known by and under the care of a village or commune health worker. All women were individually invited to participate in the project by the commune health worker in charge of her specific village. Introduction to the project and seeking informed consent for participation took place during a woman’s first prenatal visit, where a health worker described the mMom project and asked for the woman’s consent to receive SMS messages over the course of her pregnancy and participate in project evaluation activities. If she agreed, she was asked to sign a consent form and complete the preintervention survey. Her phone number and unique identity code were then added to the database, and she began to receive messages based on her estimated week of pregnancy. Once she had delivered her baby, the participant was switched to the new mother SMS program. The study was approved by an institutional review board at PHAD, which was registered with and used ethical protocols of the Office for Human Research Protections of the United States Department of Health and Human Services. Project data were subject to privacy and security protocols of the Office for Human Research Protections and the TNHD.
The mMom platform and intervention were developed over 6 months, piloted for 2 months, and implemented for 24 months. Project wrap-up occurred over the final 10 months; project data are currently under analysis, and dissemination of results is forthcoming.
The project evaluation framework included the following 4 major components: (1) initial ethnographic fieldwork, (2) pre- and postintervention surveys in intervention and control communes, (3) regular M&E site visits, and (4) extended mid-term and final evaluations. Each of the four components of the project evaluation framework were planned and designed at the project outset. As described earlier, ethnographic fieldwork was carried out in Dinh Hoa district at the beginning of the intervention development phase to learn about project feasibility and potential challenges and to further contextualize and refine project details. The team included experts (in obstetrics and gynecology, MNCH, public health, IT, and M&E) from Simon Fraser University (Canada), PHAD, National Hospital of Obstetrics and Gynecology (Hanoi), Hanoi Medical University, and TNHD. Focus group discussions and in-depth interviews were carried out with staff of the district and commune health centers, pregnant women and women with an infant aged less than 1 year, and family members of these target population women, especially their husbands and mothers-in-law.
Preintervention questionnaires were developed and piloted at the beginning of the project. All pregnant women and women with infants aged less than 1 year in 8 intervention and 4 control communes were invited to respond and provide information on their knowledge, attitudes, and practices around MNCH. The questionnaire was formulated to evaluate participants’ knowledge of food sources for specific nutrients, what types of experiences merited medical attention during pregnancy or new motherhood, and their attitudes toward MNCH; and to assess participants’ current mobile phone use habits, their family income, prior health care experiences, health insurance status, and other demographic details. When participants delivered their infant or experienced a miscarriage (for the pregnancy program) or when their child reached 1 year of age (for the new motherhood program) or when they wanted to cease participation for any reason, they were asked to be interviewed using a postintervention questionnaire. The postintervention questionnaire was formulated to mirror the preintervention questionnaire to assess changes in knowledge, attitudes, and practices over the course of the intervention. It is expected that this intervention-control survey design will allow us to control for confounding factors and learn about net effect of the mMom intervention with appropriate techniques, such as difference-in-differences analysis. Additionally, participants living in the 8 intervention communes were asked about their experiences with and attitudes toward the mMom project. Commune health center staff were trained to administer the questionnaires under the supervision of PHAD project officers. EpiInfo (Centers for Disease Control and Prevention, Atlanta, US) was used to develop data entry forms, and double-entry protocol was employed to control data entry quality.
Regular M&E trips were conducted by project management teams from PHAD, TNHD, and Dinh Hoa district health center to participating communes for monitoring the intervention’s progress and for anticipating and addressing any issues that arose. The mid-term and final evaluations were carried out in May 2014 and May 2016, respectively, and were led by mixed teams of semiexternal evaluators, who knew the project well but did not take part in implementing the intervention, and external evaluators (including MNCH and IT experts) to gain objective project insights. Qualitative data collection guidelines (focus group and interview guides,
The results of the mMom project development process were as follows: (1) the successful development of the mMom system, including the mHealth platform hardware and integration into the HMIS, the intervention plan and content, project personnel, and M&E framework; (2) the piloting and implementation of the intervention through multistakeholder partnerships as planned; and (3) the implementation of all planned M&E activities.
The mMom project’s design and research protocol aim to contribute several lessons on multilevel stakeholder partnerships and the integration of mHealth systems with existing electronic population and health record systems, which are relatively lacking in the global mHealth literature.
The mMom system focused on development through partnerships at local, provincial, and national levels to enhance the sustainability of the intervention. One of its anticipated strengths is that the project was informed by global-level frameworks, guidance, and best practices on the application of mHealth to improve MNCH outcomes, but it was also highly context-specific because of careful attention to the local relevance of the SMS message content, a piloting phase in the Dinh Hoa district, and local implementation through existing community health work systems. The project’s high level of integration with local partners, including TNHD, the Dinh Hoa district health center, and the local commune health centers that carried out daily project activities, was anticipated to present challenges because of the significant communication necessitated by this approach. To ensure effective coordination, two dedicated project officers at PHAD communicated routinely with all partners, and the project team met the TNHD, the district health center, commune health staff, and a number of participants on at least a biannual basis to discuss the progress of the intervention and share perspectives on its future. It is expected that this consistent engagement will promote partners’ commitment to the project over the longer term.
The enabling policy and technological environment in Thai Nguyen is expected to promote this intervention’s continued integration and uptake among high-level decision makers in Dinh Hoa district and at the TNHD. As previously described, Thai Nguyen province has adopted a HMIS and has increasingly applied technology in improving health service delivery over the past two decades. In terms of technological integration, Thai Nguyen is advanced relative to other provinces in Vietnam, and officials at several health system levels have engaged with and received training in health information technologies in recent years. It was expected that this material context (ie, eHealth information components that are already in place) and a social and policy context which is committed to exploring and applying mHealth to improve health outcomes, would create an environment of high readiness for a project such as mMom and would enable the project’s effective implementation.
It was also anticipated that the mMom project would be innovative and sustainable because of its integration into the local HMIS and its implementation by the local health workers at district and commune levels. Furthermore, the TNHD was engaged in comanaging and overseeing operation of the intervention from its initiation and also took ownership of the mMom system at the official end of the project period. It was expected that continuous involvement of the health centers and government bodies at the district and provincial level would support the project’s integration into current systems and will boost the likelihood of its permanent adoption.
This protocol outlines the research tools and process of the mMom project in northern Vietnam, which aims to examine the feasibility and impact of implementing an integrated mHealth intervention to improve maternal and child health knowledge and behavior among EMW living in a remote area in Vietnam. The project is likely to provide important lessons on the challenges of a highly integrated multistakeholder partnership and insights on coordination between the international project team, the provincial health department, and the district and commune health center levels, which took place as part of implementation. The project may also contribute lessons on factors that may enable or present challenges to integrating a novel component into an existing HMIS and lessons on district management of mHealth implementation. These potential contributions can provide valuable insights into currently existing gaps in the mHealth literature and are expected to support the increased integration of mHealth for MNCH projects in other contexts.
Qualitative data collection guidelines, mMom project evaluation.
electronic health
ethnic minority women
health management information system
International Development Research Centre
information technology
monitoring and evaluation
Mobile Alliance for Maternal Action
mobile health
maternal, newborn and child health
Institute of Population, Health, and Development
short message service
Thai Nguyen Provincial Health Department
World Health Organization
The mMom project wishes to acknowledge the TNHD, the Dinh Hoa district health center, all participating commune health centers, the dedicated commune health workers, the project participants and their families, and the Departments of Maternal and Child Health and Information Technologies at the Ministry of Health for their committed support to this intervention. The mMom project wishes to acknowledge the funding provided by the IDRC, and the funding and project support provided by Simon Fraser University, Canada.
None declared.