ARTICLE IN PRESS
Development and Evidence of Content Validity of the Spectrographic Vocal Assessment Protocol (SVAP) Gabriele Rodrigues Bastilha, Karina Carlesso Pagliarin, and Carla Aparecida Cielo, Santa Maria, Brazil Summary: Objective. To develop and seek evidence of content validity of a spectrographic vocal assessment protocol. Methods. Methodological study for development and validation of an assessment instrument. A broad literature search was conducted to develop and substantiate a spectrographic vocal assessment protocol. The protocol items were sent to ﬁve expert judges, speech therapists, nonauthors of the research, doctors, and clinicians, with experience in the ﬁeld of voice and spectrographic acoustic analysis, who individually analyzed the protocol items. For data analysis, the Gwet analysis and the Content Validity Ratio formula were used. Results. The Spectrographic Vocal Assessment Protocol was developed, subdivided into Broadband and Narrowband. Twenty-four protocol items had Content Validity Ratio 1.0 and 25 items had 0.6. Gwet analysis indicated substantial agreement (0.656) for broadband and almost perfect (0.848) for narrowband items. Conclusion. Evidence of satisfactory content validity was obtained in the development of the protocol. Key Words: Acoustics−Spectrography−Validation Studies−Speech, Language and Hearing Sciences−Voice.
INTRODUCTION Vocal assessment should analyze all dimensions of vocal behavior, so it is recommended to conduct a multidimensional voice assessment, considered the basis of diagnostic and therapeutic clinical reasoning.1−10 Among the existing assessments in the vocal clinic, there are auditory perceptual, self-assessment, aerodynamic measurements, glottal source acoustics, spectrographic acoustics, and visual perceptual analysis of laryngoscopy.3−6,8−9,11−13 The visual recording of sound wave analysis through vocal spectrography presents the distribution of energy in relation to frequency and time, allowing the visualization of noise recording, among other aspects. Spectral components belonging to vocal alterations may be related to different vocal qualities and characteristics of acoustic energy distribution.6,10−12,14−18 This analysis provides data that is related to the vibratory pattern of the vocal folds, the shape of the vocal tract and its changes in time, and these data vary with age, gender, vocal training, and phonation type.5,11,17,19,20 Spectrographic acoustic analysis is therefore an important part of multidimensional vocal assessment. It can be used in both research and clinical practice to characterize vocal disorders and voices of different populations, to assess patient outcome and the effect of speech therapy, and as a visual feedback of the emission.5,6,11,14−16,21,22 However it depends on the evaluator's visual analysis, which characterizes an aspect of subjectivity in this analysis. Therefore, the importance of developing and validating a spectrographic vocal assessment protocol (SVAP) was Accepted for publication December 6, 2019. From the Department of Speech-Language Pathology and Audiology, Universidade Federal de Santa Maria/UFSM, Santa Maria, Rio Grande do Sul, Brazil. Address correspondence and reprint requests to Gabriele Rodrigues Bastilha, Department of Speech-Language Pathology and Audiology, Universidade Federal de Santa Maria/UFSM, Av. Roraima no 1000, Prédio 26, 97105-900 Santa Maria, Brazil. E-mail: [email protected]
Journal of Voice, Vol. &&, No. &&, pp. &&−&& 0892-1997 © 2019 The Voice Foundation. Published by Elsevier Inc. All rights reserved. https://doi.org/10.1016/j.jvoice.2019.12.008
realized, covering aspects that are fundamental in this type of analysis, based on the studies found in the literature. These studies bring different approaches to the assessment of spectrographic images,1,6,8,12,18,23−27 which makes the spectrographic vocal assessment even more subjective and hinders the analyses and, consequently, the standardization of results and their understanding by the clinical speech therapist. Thus, the validation of a protocol for spectrographic vocal assessment could provide standardization of assessments, interpretation of their results, and help speech therapists in both clinical practice and scientiﬁc research. Assessments based on speciﬁc protocols allow the clinician to act in a planned manner, to document procedures, solidifying and sustaining with scientiﬁc evidence the clinical practices. Validation of the protocol could also facilitate peer discussion of the analyzed spectrographic vocal aspects.11,16,28 Regarding the protocol development and validation process, there are a number of methodological criteria that must be strictly followed. The elaboration process must be carefully planned so that the instrument can be able to assess and provide data relevant to what it proposes to investigate.28−30 In this sense, evidence of validity allows for more accurate decision making and increases scientiﬁc rigor in interpreting instruments.28,31 One of the types of validity, content validity requires the test to present a representative sample of a ﬁnite universe of behaviors. To enable a content-validated test, test speciﬁcations must be made prior to constructing the items.31,32 Content validity is veriﬁed from the analysis of individuals with expertise in the area about the semantic understanding of the items that make up the instrument (clarity) and the relevance/pertinence of these items.12,31 To standardize the aspects that are important in spectrographic analysis and considering the lack of validated instruments in this area, this study aimed to develop and seek evidence of content validity of a SVAP.
ARTICLE IN PRESS 2
Journal of Voice, Vol. &&, No. &&, 2019
METHODS Study design This is a methodological research, approved by the Ethics Committee on Research with Human Beings of the home institution with the number 1.570.886. Participants To select the volunteers participating in this research, a search was made through the Lattes CNPq database of Voice specialists from different regions of Brazil. Aspects such as formation, production (articles and abstracts), and projects with the subject spectrographic acoustic analysis were considered. Initially, eight judges were selected, but three were excluded because they had no experience with spectrographic analysis speciﬁcally, only with acoustic analysis. Finally, the judges selected were ﬁve speech therapists, nonauthors of the research (doctors and clinicians), with experience in the ﬁeld of voice and spectrographic vocal analysis of at least ﬁve years, who independently analyzed the items of the proposed protocol.12,32,33 The judges were invited to participate by e-mail and signed an informed consent form. Procedures This research aims to present the development and content validation process of an assessment instrument. This process is subdivided into the development of the instrument, with extensive literature review for the theoretical foundation of the aspects to be assessed by the instrument, and content validation.31 Therefore, the methodological procedures performed were described from this subdivision. Protocol development The SVAP was constructed based on the literature, considering the aspects analyzed in spectrographic vocal assessments and encompassing the largest possible number of parameters.1,6,13,15,16,18,22−26,34−37 SVAP has been used by our research group since the late 1990s. Several spectrographic vocal aspects not validated in the literature and currently used in clinical and speech therapy research were identiﬁed and included in a single protocol. Thus, the SVAP was elaborated, consisting of two parts: Broadband Spectrography (BS) and Narrowband Spectrography (NS). Each part with its respective aspects/parameters to be assessed, as well as general written instructions to the evaluators for ease of understanding. To ﬁll the SVAP, a linear analog scale with a score ranging from 0 to 10 cm is used. The assessment of each parameter should be performed by marking a vertical line at any point on the millimeter ruler, according to the observed gradation.2,4,6,8,22,37 In BS, the following parameters are assessed: formant tracing color intensity (1st Formant-F1, 2nd Formant-F2, 3rd Formant-F3, and 4th Formant-F4), of low, medium, and high frequencies and the whole spectrography vocal; tracing regularity at whole vocal spectrogram and at low, medium, and high frequencies; deﬁnition, regularity, and bandwidth of
F1, F2, F3, and F4; and antiresonance/damping immediately above F1, at whole vocal spectrography and at low, medium, and high frequencies. In the NS analysis, the following are assessed: tracing color intensity, deﬁnition, regularity, and number of harmonics, presence of subharmonics, presence of noise and replacement of harmonics by noise at low, medium, and high frequencies and at whole vocal spectrography.6,8,12,13,15,16,22−26,34,36−39 For purposes of parameter analysis, low frequencies below 1500 Hz, medium frequencies between 1500 and 3000 Hz and high frequencies above 3000 Hz are considered.5,23,25,35,40 The ﬁrst part of the protocol consists of ﬁve items, with their respective subitems, related to BS, detailed below: 1. Tracing color intensity of the four F, low, medium, and high frequencies, and in vocal spectrography as a whole: The tracing color intensity is related to the acoustic signal sound pressure. In spectrography, the degree of darkening present in the trace should be considered, which may vary from black (corresponding to 10 cm), indicating strong signal sound pressure, to light gray, indicating weak sound pressure, or white, suggesting silence. Reduction in the tracing color intensity of some F may suggest nasality, such as F1 or F3, being considered as a result of the addition of damping characteristics on the nasal cavity wall surfaces. Tracing color intensity should be analyzed at each of the four Fs, as well as at low, medium, and high frequencies and in vocal spectrography as a whole.5,26,27,41−43 2. Deﬁnition and regularity of the four Fs: The visible, well-demarcated and symmetrical Fs along their temporal representation are considered to be well-deﬁned and regular (corresponding to 10 cm). F that are poorly visible, poorly demarcated, and/or asymmetrical may be considered with an altered deﬁnition and F unidentiﬁable are considered absent with 0 cm marking.22,27,35 3. Tracing regularity at low, medium, and high frequencies and in spectrography as a whole: This aspect deals with the continuity and stability of the vertical striation tracing and relates to the presence of noise or aperiodicity in the sound signal. Thus, a continuous and regular tracing corresponds to 10 cm (total tracing regularity) and a tracing dominated by oscillations, erasures or other types of irregularities corresponds to the total irregularity (0 cm).25,27 4. Bandwidth of the four Fs: The Fs are classiﬁed according to their bandwidth, with 10 cm being fully increased and 0 cm being fully decreased F band. F bandwidth represents the effective frequency range of resonator response and increased bandwidth may suggest hypernasality.13,27,44 The increase in F113 and F344 bandwidth is related to the auditory-perceptive characteristic of breathiness. 5. Anti-resonance/damping immediately above F1, at low, medium, and high frequencies and in spectrography as a whole: Described in the literature as an acoustic mufﬂing due to sound dampening, this aspect may be
ARTICLE IN PRESS Gabriele Rodrigues Bastilha, et al
Development and Evidence of Content Validity
related to the direction of sound to the nasal cavity. It corresponds to 10 cm in the domain of erasure spectrography (blank or much lighter than the rest of the spectrography) and 0 cm no identiﬁed erasure.27,41,43 The second part of the protocol consists of six items and their respective subitems related to NS: 1. Tracing color intensity of low, medium, and high frequency and in vocal spectrography as a whole: As already mentioned, tracing color intensity refers to sound pressure. The degree of darkening should be considered, which may vary from black (corresponding to 10 cm), indicating strong signal sound pressure, to light gray, indicating weak sound pressure, or white, indicating silence.12,16,26,27 2. Presence of noise at low, medium and high frequencies and in vocal spectrography as a whole: Noise appears in spectrography as a shaded or dotted image and is associated with the aperiodicity of the sound signal. Shading may vary in color intensity from black (corresponding to 10 cm), indicating more noise, to light gray indicating less noise or white, indicating no noise.5,16,26,27,37,40 3. Replacement of harmonics by noise at low, medium, and high frequencies and in vocal spectrography as a whole: Noisy voices can have their harmonics replaced by noise at NS and this can happen especially at high frequencies, showing a voice poor in vocal projection. The absence of harmonics above 3000 Hz in altered voices occurs because aperiodic sound waves are not uniformly ampliﬁed, which prevents their multiples from being represented as harmonics in the graph. Thus, the region above 3000 Hz may appear darkened only by the presence of noise.12,40 10 cm for total replacement of harmonic by noise is considered. 4. Deﬁnition and regularity of harmonics at low, medium, and high frequencies and in vocal spectrography as a whole: In a normal voice, harmonics (horizontal lines) are expected to be well deﬁned and continuous, showing adequate glottal closure, absence of air escape to phonation, adequate sound pressure, and periodicity in mucosal vibration, showing a regular and well deﬁned tracing (corresponding to 10 cm). When this does not occur, there is a tracing with undeﬁned harmonics, with cuts and or irregularities.16,26,27,35,36,40 5. Number of harmonics at low, medium, and high frequencies and in vocal spectrography as a whole: This relates to the quantity of harmonics present in spectrography in the analyzed frequency range. The 10 cm marking is considered for the complete ﬁlling of the spectrographic image by harmonics and 0 cm for the complete absence of harmonics in the NS.26,37 6. Presence of subharmonics in low, medium, and high frequencies and in vocal spectrography as a whole: This is a tracing present between two consecutive harmonics, showing a source of vibration beyond the vocal folds or signifying the presence of noise in the sign. They can be
a partial or total duplication of harmonics. In this item, 10 cm corresponds to the presence of subharmonics at a whole spectrography and 0 cm to the complete absence of subharmonics in the NS image.16,23,26,34−37 Content validation The ﬁve judges received the SVAP by email and were instructed to carefully analyze their items, classiﬁed each item of the instrument as "essential" or "not necessary," in others words, if these were needed to be included or not. Each item of the instrument presented a brief orientation (Appendix A) and the judges relating their inclusion with concepts and theoretical considerations. Data analysis was performed by calculating the Content Validity Ratio (CVR) for each item. CVR was obtained by the formula CV = (ne N/2)/(N/2), where he is the number of judges who considered the item as adequate and N represents the total number of judges. Also, to verify agreement among judges, Gwet's ﬁrst-order agreement coefﬁcient (AC1) statistical calculation was used.45 AC1 agreement was measured by Landis and Koch criteria46 where >0.8 is considered almost perfect, 0.61−0.8 is substantial, 0.41−0.6 is moderate, 0.21−0.4 is regular, and