https://doi.org/10.33698/NRF0316-Sonia R.B. D`Souza ,Vijay Kumar,Leslie Edward Lewis
Abstract: The aim of this study was to validate the content and check the reliability of a tool to assess the behavior of the preterm infants aged 28 – 33 weeks of gestational Age following a Developmental Supportive Care (DSC) Intervention. A multidisciplinary panel of seven national and international experts was selected to assess, review, and suggest modifications for the tool. The tool had two subsystems. One was the autonomic/visceral system consisting of four domains namely colour, respiration, visceral signs and neurophysiological responses. The second was the state and attention-interaction system, consisting of three domains namely state regulation, orientation to auditory stimulus and alertness. A validity questionnaire was keyed to the individual items clustered under the two subsystems and the initial draft of the tool was mailed to the content experts along with the objectives, aim and the blueprint of the tool. Experts were from the field of Pediatrics, Neonatology, Developmental Psychology, Perinatal nursing and Neonatal nursing. Data were analyzed using descriptive statistics. The Content Validity Index (CVI) for each item as well as the overall CVI were calculated. All the seven experts completed the validity questionnaires. The experts generally agreed that the items of the initial draft were clearly worded, appropriate and relevant. The tool obtained a high overall CVI score of 0.95, yielding a valid tool for assessing behavior of the preterm infants. To check the reliability of the tool, two raters observed 40 preterm infants for duration of 30 minutes following the provision of 30 minutes of the DSC intervention. The inter-rater reliability was checked using Intra class correlation coefficients (ICC). Calculation was done using SPSS (Ver.15.0), South East Asia, Bangalore for raw scores obtained by two independent observers for the two subsystems namely the autonomic and state and attention-interaction systems and their domains in the tool. The ‘R’ value obtained for the autonomic subsystem was 0.96 whereas the ‘R’ obtained for the state and attention-interaction subsystem was 0.92, which indicates high reliability for both the subsystems of the tool. The findings reported suggest that the tool has potential for use as both a clinical as well as a research instrument to assess behaviors of preterm infants who are provided DSC interventions.
Keywords:Behavior of preterm infant, content validity index, reliability
Correspondence at:Mrs. Sonia R.B D’Souza Associate Professor, Department of OBG Nursing, Manipal College of Nursing Manipal, Manipal University, Manipal, Karnataka
Introduction:There has been improved survival of preterm infants of very young gestational age due to the advancing technology and improved evidence informed perinatal care. In the bargain of survival, preterm infants are exposed to numerous hazardous environmental stimuli especially excessive sound in the NICU.To minimize the effects of the environmental stimuli, Developmental Supportive Care (DSC) interventions (E.g. music therapy, light touch, massage, vestibular stimulation etc.) are encouraged to be provided for the preterm infants admitted to the NICU. DSC interventions are employed in the NICU for providing the necessary stimulation and for alleviating stress experienced by these infants. However, it is also necessary that the behavior of the preterm infants be observed following the DSC interventions.As, Als1 states that if the interventions meant to provide stimulation to the preterm infants are appropriate for the preterm infant in terms of complexity, intensity and timing, the preterm infant has strategies available to him to actively move towards that stimulation, take it in, and make use of it for his own development. If stimulation on the other hand is currently inappropriate for the infant, the infant is thought to avoid and defend against that stimulation. If the level of input and information is currently appropriate for the infant, he maintains a balanced and well-regulated behavioral modulation.1 In the recent times, worldwide there is a paradigm shift in the focus of care and the interventions employed for the preterm infants from mere survival to optimal development. DSC interventions, not only support the NICU medical care but also contribute to the optimal development of preterm infants.2 However, it is essential to know whether this care is appropriate to the preterm infants because it is necessary to identify stressors experienced by preterm infants during these interventions. The stressors experienced are communicated by preterm infants through their behavior. Each behavior has a meaning assigned to it, though the behaviors exhibited by preterm infants may not be as clear and distinct as term infants. A valid and a reliable tool was thus necessary to assess the behavior of the preterm infants as well as to determine the behavior modulation any DSC interventions could bring about in the preterm infants.
Objective:To develop a valid and a reliable tool for assessing behavior of the preterm infants 28 – 33 weeks of gestational Age after provision of DSC interventions in the NICU.
Materials and Methods:For the present study, the tool development proceeded through a development stage. To fulfill this, an extensive review of the literature was undertaken to identify articles pertaining to assessment of behavior of the preterm infants. The domains of the tool were developed based on the Als Synactive model1of neonatal behavioral organization. The tool consists of two subsystems i.e, the autonomic/visceral subsystems and state organization and attention-interaction subsystem. The tool and the scoring of the tool is provided in table 1 & 2.
Table 1: Assessment of preterm infant behavior :autonomic/ visceral system
| Score | Colour | Respiration | Visceral | Neurophysiological responses |
| 0 | Pale, Cyanotic | Gasping, frequent apnea, unstable respiratory rate | Vomits feed, feed intolerance | Flaccid on stimulation |
| 1 | Pink but changes rapidly with slow recovery, not returning to good color | Occasional apnea, unstable respiratory rate | Bowel Movement grunt and strain | Abnormal jerks, Twitch |
| 2 | Pink | Regular, stable respiratory rate | Gastrooesophageal Reflux (GER) Nil | Stable |
Table 2: Assessment of preterm infant Behavior: State system and attention-interaction system
| Score | State regulation | Orientation to auditory stimulus | Alertness |
| 0 | Intense crying which is rhythmic with irregular breathing | Does not focus on or follow stimulus | Rarely or never responsive to direct stimulation |
| 1 | Active awake state with infant fussing but not crying but stressed and hyper alert | brightness with stimulus, may focus and follow briefly with jerky
eye movements |
When alert, responsivity brief and variable, may be delayed |
| 2 | Awake alert | Focuses on stimulus and follows with smooth continuous head movement | Always alert in best periods, stimulation always elicits alerting and orientating. |
The tool has a point system scoring i.e., from zero to two for the autonomic/ visceral items expressed in domains viz. colour, respiration, v i sceral s i gnals and neurophysiological responses. The state and attention/interaction system has items to assess preterm neonates in the early postnatal period with domains like state regulation, alertness and orientation to auditory stimulus. In the autonomic/visceral subsystem, the maximum total score for each domain in the tool is 8. If the infant obtains a score of >5-8 for all the four domains in the subsystem, the preterm infant is considered to show ‘normal behavioral response’.But, if the infant scores 2-4 then it is considered to be showing ^suspected abnormal behavioral response’ and if scores obtained <1 then the infant is showing ^definite abnormal behavioral response’. Similarly, in the state system and attention-interaction subsystem the maximum total score for each domain of the tool i.e. state regulation, orientation to auditory stimulus and alertness is 6. If the preterm infant obtains a score of >4-6 for all the three domains in the subsystem, the preterm infant is considered to show ‘normal behavioral response’, but if the infant scores 2-3 then it is considered to be showing ^suspected abnormal behavioral response’ and if scores obtained <1 then the infant is showing ‘definite abnormal behavioral response’ respectively.
Validation of tool: Following this, the research team proceeded with the judgment-quantification stage where a panel of experts was requested to evaluate the tool for item content, clarity of items and comprehensiveness of items and rate items of relevance according to the content domain. For this study, an interdisciplinary panel of seven national and international experts was chosen by the research team, which included researchers and expert clinicians in Pediatrics, Neonatology, Developmental Psychologist, Perinatal nursing and Neonatal nursing. Cover letters explaining the purpose of the tool along with background, aim of the study and instructions were provided to the panel of experts. Informed consent was obtained from the experts and the researcher explained the process to the panel of experts to ensure understanding of the process. The panel was asked to review the items in the tool. All the respondents completed all questions on the questionnaire. Some respondents provided narrative feedback to some questions posed. The content experts (2 women, 5 men) had an average of above 10 years of experience in their respective disciplines (Pediatrics, Neonatogy, development psychology, perinatal nursing and neonatal nursing).Subsequent to this, the completed questionnaires by experts’ were inferred. There were no major modifications suggested by the experts for the initial draft of the tool. So, the tool had only one round of content validity. Further, the research team tallied the results of the experts based on the degree to which the experts agreed on the accuracy, relevance and appropriateness of the content clustered under the domains of each of the two subsystems of the tool. Subsequently, the Content Validity Index (CVI) was calculated for determining content validity agreement between the experts for all the domains of the tool. (Table 3 & 4)
Table 3: Content Validity Index (CVI) of the autonomic/visceral subsystem of the tool in terms of accuracy, relevance and appropriateness n=7
| Score | Domains | Number of experts agree | Number of expert did not agree | CVI |
|
0 1 2 |
Colour
Pale, Cyanotic Pink but changes rapidly with slow recovery Pink |
6 7 7 |
1 0 0 |
85.71 100 100 |
|
0 1 2 |
Respiration
Gasping, frequent apnea, unstable respiratory rate Occasional apnea, unstable respiratory rate Regular, stable respiratory rate |
7 6 7 |
0 1 0 |
100 85.71 100 |
|
0 1 2 |
VISCERAL
Vomits feeds, feed intolerance Bowel movement grunt and strain Gastro-Oesophageal Reflux (GER) Nil |
7 6 7 |
0 1 0 |
100 85.71 100 |
| NEUROPHYSIOLOGICAL RESPONSES | ||||
| 0 | Flaccid on stimulation | 7 | 0 | 100 |
| 1 | Abnormal jerks, Twitch | 6 | 1 | 85.71 |
| 2 | Stable | 7 | 0 | 100 |
Table 4: Content Validity Index (CVI) of the state and attention-interaction subsystem of
|
the tool in terms of accuracy, relevance and appropriateness
n=7
Content Validity Index (CVI) of the tool:The CVI from the panel of experts for the domains of the autonomic/visceral subsystem of the tool is shown in table 3. The CVIs for the four domains of the autonomic/visceral subsystem were above 0.80 indicating adequate agreement; and for eight domains the CVIs of 1.00 indicate 100 percent agreement between the experts or raters. The overall CVI from the panel of experts for the autonomic/visceral subsystem yielded a 0.95. Since the overall CVI is above 0.90, it indicates good agreement between the experts for all the domains of this subsystem of the tool.The CVI from the panel of experts for the domains of the state and attention- interaction subsystem of the tool is shown in table 4. The CVI for two domains of the state and attention-interaction subsystem was above 0.80 indicating adequate agreement; and for six domains the CVI of 1.00 indicated 100 percent agreement between the experts. Even though the CVI for one domain on alertness in the state and attention- interaction subsystem of the tool was 71.43, since two experts disagreed on ‘complexity of the wording’ of the domain, the overall CVI from the panel of experts for the state and attention-interaction subsystem yielded a CVI of 0.94. Since the overall CVI is above 0.90, it indicates good agreement between the experts.The disagreement between the experts for one domain namely ‘alertness’ was due to the ‘complexity of the wording’ of the rubric. The experts suggested that the rubric should be reworded as ‘active alert’ instead of the initial wording of the rubric i.e., ‘Always alert in best periods/stimulation always elicits alerting and orientating’. The suggestion was incorporated in the final form of the tool and the rubric was reworded as per suggestions of the experts. Further the overall CVI for both the subsystems of the tool yielded 0.95 showing good agreement between the experts. A high overall CVI of 0.95 demonstrates the validity of the tool. Lynn3 has stated that the CVI obtained from a panel of six experts or more should be more than 0.70. The research team also maintained that the CVI obtained for all domains of the subsystems of the tool should be no lower than 0.70 for deeming the tool as acceptable and valid in terms of content validity.
Reliability of the tool:
The purpose of the present study was also to examine systematically the reliability of the tool developed to assess preterm infants following provision of the DSC intervention. For measuring reliability, a group of 40 preterm infants of 28 – 33 weeks of gestation age and whose chronological age was first week of life were observed for 30 minutes after provision of DSC intervention for 30 minutes. The two raters, each of them were registered nurses (both female) and were working in the Neonatal Intensive Care Unit (NICU) as staff nurses and each having an average of eight years of clinical working experience in the NICU. The raters simultaneously observed the preterm infants after provision of DSC intervention for a period of 30 minutes to provide an estimate of inter-rater reliability. Approval was taken from Manipal University Ethical Committee prior to the commencement of the study. Criteria for inclusion of the preterm infants required one or more of the following: preterm infants who were 28-33 weeks of gestational Age, belonging to ‘weight for gestational age’ like appropriate for gestational age (AGA) and small for gestational age (SGA), cared for in the open care system with an overhead radiant warmer and consented to be included in the study by proxy consent i.e. parental informed written consent. Forty preterm infants, who participated in this study, were admitted to the tertiary level NICU at the time of assessment. Parental informed proxy consent was taken before commencement of the study.Criteria for exclusion were: preterm infants who were belonging to ‘weight for gestational age at birth’ i.e., large for gestational age (LGA), having congenital malformations, dysmorphic features, chromosomal abnormalities and surgical complications, preterm infants receiving incubator care, having congenital sepsis (confirmed by blood culture at CA of 1-3 days), exposed to Chorioamnionitis (confirmed in the antenatal record), having Central Nervous System (CNS) infection and any other high-risk condition identified by the NICU staff or the preterm infant’s physician. All preterm infants were assessed during the first week of life.Two raters participated in the collection of the reliability data. The research team trained both the raters for examining and observing the behavior of the preterm infants using the tool. One rater was designated as the primary rater and the other rater was designated as the secondary rater. Each rater rated the tool independent of the other rater for the duration of 30 minutes of observation. When the observations were done on 40 preterm infants during the first week of life, following the provision of DSC intervention, the research team ensured that the raters did not discuss the scoring with each other as the test proceeded. The observations were performed following 30 minutes of provision of DSC intervention. The raters observed the preterm infants kept in the open care system following the provision of DSC intervention i.e. for 30 minutes and recorded their observations. The DSC intervention in this study was recorded Indian Classical music which was provided for duration of 30 minutes.The Tool could be administered and scored in less than 15-20 minutes, which was well within the designated observation period set by the research team viz. 30 minutes duration after provision of 30 minutes of DSC intervention.
Results: Reliability of the tool:
In this study, the consistency estimates of Interrater or inter-observer reliability was the intra-class correlation (R) which provides an estimate of the relation between the scores of two raters for the paired scores on observation (inter-observer reliability).The Pearson product- moment correlation coefficient calculates the degree of consistency between independent raters. Values approaching +1 or -1 indicate that the raters are following a consistent pattern, whereas values close to zero indicate that it would be almost impossible to predict the rating of one rater given the rating of the other rater. The minimal acceptable level of ‘R’ is 0.60.4 Calculation of ICC coefficient ‘R’ for the present study was done using SPSS (Ver.15.0), South East Asia, Bangalore for raw scores obtained during observation of preterm infants following the DSC intervention by two independent observers for the two subsystems namely the autonomic and state and attention- interaction systems and their domains in the tool. Following tabulation of the raw scores for each of the domains and their subsystems on a contingency table, the ICC was calculated. The ‘R’ value obtained for the autonomic subsystem was 0.98 whereas the ‘R’ obtained for the state and attention- interaction subsystem was 0.92, which indicates high reliability for both the subsystems of the tool.
Discussion: D S C i n t e r v e n t i o n s p r o v i d e developmental support to the preterm infants. But, to ensure that the DSC i n t e r ventions help promotion o f developmental support to the preterm infants and not induce any stress to the preterm infant, a valid and reliable tool was necessary. A valid and a reliable tool for assessing the behavior of preterm infants is an absolute necessity. The tool developed in this study assessed the preterm infants following provision of a DSC intervention. The present study shows that the tool developed is a valid and reliable tool for assessing preterm infants after provision of a DSC intervention.The present tool was developed based on certain domains of ^The Assessment for Preterm Infants’ Behavior'(APIB) developed developed by Brazelton6,which assesses the behaviors of term infants. Certain domains of the state organization and the attention- interactionsystem of the tool which includes state regulation, orientation to auditory stimulus and alertness, were adapted from an assessment program to assess preterm neonates in the early postnatal period.6 This was the first tool of its kind, which was specifically developed to assess immediate behaviors of the preterm infants following a DSC intervention. All in all, the tool developed, demonstrated good content validity and reliability. But, the reliability testing of the tool was done on only 40 preterm infants after provision of the DSC intervention. Clearly, a study with a larger sample size is needed to determine if the reliability trends obtained for the subsystems of the tool in the present study are consistent and stable when using the tool further.Another important limitation of this study, however, was the fact that though interdisciplinary panel of experts were chosen to validate the content of the tool, the content experts were predominantly from the field of Pediatrics and Neonatology whereas there was little representation of experts by Al et al.5 APIB was developed for use with from other fields of health care providing preterm and ill full-term newborn infants to identify both neurobehavioral competencies and threshold to stress. The tool for assessment of preterm infant behavior in this study does not study neurobehavioral comprentencies, but determines stress behaviors as well as self regulation behaviors of preterm infants following a DSC intervention. The research team had also reviewed and adapted some domains of behavioral assessment from the Neonatal Behavioral Assessment Scale (NBAS) care to the preterm infants, since there were four clinicians belonging to Pediatrics and Neonatology whereas one each representing perinatal nursing, neonatal nursing and developmental Psychology. The experts from the field of Pediatrics and Neonatology who were chosen could have primarily given their opinions based on their medical expertise. This shortcoming minimizes the generalizability of these opinions. Finally, the value of the experts’ opinions about the tool would have strengthened considerably, had the experts been given the opportunity to actually administer the tool rather than to merely review the tool, though this is not essentially needed for validating the tool.The findings reported suggest that a valid and reliable tool for neonatal assessment was prepared which has potential for use as both a clinical and a research instrument by Pediatricians,
References
- A synactive model of neonatal behavioral organization: Framework for the assessment and support of neurobehavioral development of premature infants and their parents in the environment of the NICU. In:Sweeney JK. Physical and Occupational Therapy in Pediatrics. 6thed. New York. Haworth Press; 1986.
- Symington A, Pinelli Developmental care for promoting development and preventing morbidity in preterm infants. Cochrane Database of Systematic Reviews [Internet] 2006. Issue 2. Art. No.: CD001814. DOI: 10.1002/14651858.CD1814.pub2.
- Ly n n M . D e t e r m i n a t i o n a n d Quantification of Content Nursing Research.1986; 35: 382–85. Neonatologists, Neonatal nurses, Perinatal nurses, Developmental Psychologists, Pediatric physical therapists, and other health professionals who are working with preterm infants aged 28 – 33 weeks of gestational age subjected to DSC interventions in the NICU.
4.Multon KD, Interrater reliability: Encyclopedia of Research 2012; A v a i l a b l e f r o m : http://www.bibb.k12.ga.us/cms/lib01/GA0100 0598/Centricity/Domain/105/Interrater%20Rel iability%20Article_1.pdf
5.Als H, Lester BM, Tronick E, Brazelton Toward a research instrument for the Assessment of Preterm Infants’ Behavior (APIB). In: Fitzgerald HE, Lester BM, Yogman MW, editors. Theory and research in behavioral Pediatrics. New York. Plenum; 1982.
6.Brazelton Neonatal Behavioral Assessment Scale: Clinics in developmental medicine. London: Spastics International; 1984.
7.Carmichael K, Burns Y, Gray P, O’Callaghan Neuromotorbehavioural assessment of preterm infants at risk for impaired development. Aust J Physiother. 1997; 43: 101-7.