Frequently Asked Questions

Question 1

Who is eligible to participate in NCI?

Accepted Answer

The goal of National Core Indicators is to understand the performance of state service delivery systems. For NCI-IDD: In-Person Survey (IPS) includes adults who receive at least one publicly funded service (not including case management) from developmental disability services. Family Guardian Survey (FGS) includes families with an adult family member (18+) who does not live in the family home and receives one service. Adult Family Survey (AFS) includes families with an adult family member (18+) who lives in the family home and receives one service. Child Family Survey (CFS) includes families with a child (under 18) who lives in the family home and receives at least one service. For NCI-AD: Adult Consumer Survey (ACS) includes people who receive long-term services and supports defined as at least one active service (not including case management) at least twice a week.

Question 2

How do you sample participants for NCI?

Accepted Answer

NCI is a survey, not a census. For NCI-IDD: States select samples based on which populations they want to examine. Each state attempts to complete 400 or more surveys, which guarantees valid comparisons across states with a 95% confidence interval and a margin of error of +/- 5%. States use simple random sampling, stratified sampling, or proportionate sampling techniques. Family surveys are administered by selecting a random sample of at least 1,000 families. For NCI-AD: States select samples based on which populations they want to examine and determine which long-term services and supports programs to sample from. Each state attempts to complete 400 or more surveys using similar sampling techniques.

Question 3

How does NCI collect background information about demographics and services people receive?

Accepted Answer

Background information includes details about a person's characteristics and the services they receive. For NCI-IDD IPS, sources include case management records, provider records, Managed Care Entity records, state DD data systems, Medicaid billing records, and Individual Service Plans. Information includes residential details, demographics, guardianship status, diagnoses, healthcare utilization, employment details, and funding sources. For Family Surveys, all background information is reported by the responding family member. For NCI-AD ACS, sources include case management records, provider records, Medicaid billing records, eligibility records, and state data systems, covering funding sources, services received, self-directed supports, and demographic information.

Question 4

How does NCI protect the privacy of participants?

Accepted Answer

NCI is committed to transparency and confidentiality. Data is collected through HSRI's proprietary Online Data Entry System Application (ODESA), which has several layers of security including access control, authentication and authorization controls, encrypted data storage and transmittal, and physical security for data servers. Data analysis files never include personal health information or personal identifiers. When data is shared with collaborators, all data is deidentified by removing information about state of residence.

Question 5

How does NCI determine whether a proxy is needed?

Accepted Answer

The proxy determination process involves asking participants sample questions. Based on responses, the surveyor decides whether a proxy is needed. If a participant cannot answer questions validly, proxy-allowed questions are asked. The use of proxy participants is limited to questions that are observable and/or measurable. When administering proxy-allowed questions, proxies can respond but the surveyor confirms responses with the person who uses services. Service coordinators are not allowed to respond as proxies. If both the participant and proxy answer a question, the participant's answer is recorded if deemed reliable. Surveyors track who responds to each question and the proxy's relationship to the participant.

Question 6

Does NCI require consent or assent from participants?

Accepted Answer

Every state works with their own legal departments to determine state-specific consent requirements. NCI shares recommended resources and guidance on consent, including using plain language. Some states also work with Institutional Review Boards (IRBs) to ensure that state-specific consent and survey administration protocols are in line with rules about human subjects research. Considerations include legal guardians and documenting consent.

Question 7

How does NCI work to hear directly from people with disabilities and older adults who use services?

Accepted Answer

NCI uses several methods to enable equitable participation: Questions are asked directly in face-to-face interviews in comfortable settings. Surveyors ask questions in an open-ended fashion and summarize responses. Survey tools are routinely revised based on feedback to make questions accessible. Surveyors receive rigorous, standardized training. Several states use peer-surveyors with disabilities. NCI-IDD surveys have been translated into Spanish and Chinese, and NCI-AD into Spanish. Some states translate into additional languages. Trained interpreters are offered when needed. People with speech disabilities are offered accommodations including interpreters, ASL, speech-generating devices, and augmentative and alternative communication.

Question 8

What evidence is there that NCI is valid and reliable?

Accepted Answer

Researchers both internal and external to NCI have examined the psychometric properties of the various surveys, including individual items and multi-item scales. For NCI-IDD, validity has been established through face and content validity and criterion validity. Reliability has been demonstrated through inter-rater reliability, inter-unit reliability, multi-item scales, internal consistency, and principal components analysis. For NCI-AD, face and content validity have been established, and inter-rater reliability has been demonstrated. NCI is in the process of creating a manual of psychometric properties that will compile results from individual studies and the process for risk adjustment.

Question 9

How does NCI ensure all surveyors are using the same approach for survey administration?

Accepted Answer

The NCI national team administers standardized trainings for surveyors throughout the year to ensure uniform application of the survey. States employ various individuals such as state staff, private contractors, advocacy organizations, and students. A participant's service provider, relative, personal case manager, or close contact is not allowed to administer the survey. Training includes asynchronous and synchronous sessions with in-depth review of the NCI program, survey administration protocol, and the survey tool, taking approximately 6 hours. Surveyors must pass a knowledge check with at least 80%. Many states require new surveyors to complete practice surveys, achieve acceptable inter-rater reliability scores, and shadow experienced surveyors.

Question 10

How does NCI measure quality of life?

Accepted Answer

NCI's framework for individual quality of life is informed by the work of Dr. Robert Schalock and others around domains including emotional well-being, interpersonal relations, material well-being, personal development, physical well-being, self-determination, social inclusion, and rights. NCI measures outcomes categorized into three major areas: individual outcomes, family outcomes, system performance, and health, welfare, and rights. In addition to individual level outcomes, NCI examines system-level quality through annual measurement efforts, including developing measures of quality of life that can be examined in aggregate for people using the LTSS service system.

Question 11

How can NCI data be used?

Accepted Answer

States are encouraged to use NCI with other state data sources to examine system performance compared to national benchmarks. This can inform strategic planning, policy reports, and quality improvement initiatives. Many states share reports with Quality Councils, community partners, and service users. NCI data can be used to: filter or stratify reporting and examine for inequities by population (race, ethnicity, sex, age, rural/urban status, disability, language); look at year over year comparisons to track trends; examine alignment to and progress towards compliance with federal and state level policies and regulations such as the HCBS Final Settings Rule; and support value-based purchasing or value-based payment for Medicaid LTSS services.

Question 12

Why does NCI collapse responses for some of the questions in reports?

Accepted Answer

NCI reports show the percent of people that report having a particular outcome. Some responses include 'Yes,' 'In-between'/'Maybe'/'Sometimes,' or 'No.' Sometimes it makes sense to report just the percent that say 'Yes,' and sometimes it makes sense to combine 'Yes' and 'Maybe' to see the bigger picture. For example, when asking if people got to choose where they live, reporting just those who chose would fail to understand people who may have a guardianship and cannot make the decision but did provide input. Decisions about combining responses are made to measure people who have the opportunity to participate in things like making decisions.

Question 13

Is NCI data available for secondary analysis?

Accepted Answer

Participating states own their data, and de-identified NCI data is available for secondary analysis for a small fee. Those interested in learning more about the data sharing process can reach out to Dorothy Hiersteiner or Stephanie Giordano.

NCI-AD and NCI Legacy websites temporarily unavailable