For individuals with autism, a trip to the clinic — an unfamiliar place with strange people and instruments — can be exceedingly stressful. And as many genetic databases for autism include tens of thousands of participants, it is increasingly challenging for researchers to do in-person assessments for each individual.

On 14 February, the Simons Foundation Autism Research Initiative, SFARI.org’s parent organization, hosted a workshop to explore the benefits and pitfalls of using online tools to help collect data from individuals with autism as an alternative to bringing families into clinical centers.

Online assessments have a number of advantages. They don’t incur travel costs, and they can make it easier for families to participate in research. Also, researchers can immediately access the data. However, there is also the concern that data collected at home and through questionnaires is less reliable than traditional clinical assessment, and may not be representative of the wider population.

Online communities can help researchers identify potential study participants, says Paul Law, director of medical informatics at the Kennedy Krieger Institute in Baltimore. For example, the Interactive Autism Network (IAN), of which Law is the principal investigator, is a large online registry that aims to connect individuals who have autism and their families with researchers. To date, more than 43,000 individuals have joined IAN and it has recruited participants for more than 450 studies.

IAN collects parent- and self-reported data in a cost-effective, scalable manner. IAN is also launching a project called IAN Genetics that will collect blood samples for DNA analysis from interested participants.

For one ongoing IAN project, researchers are using IAN to examine whether omega-3 fatty acids alleviate hyperactivity in children with autism. The researchers recruited 57 children who had a teacher willing to participate (one of the criteria for the trial). For those 57 individuals, parents and teachers have completed every assessment provided to them so far.

Large research projects like IAN can quickly become complex, however. To address this, Law and his colleagues have developed a tool called ORCA, or Online Clinical Research and Assessment, to help researchers organize information.

Online characterization, or phenotyping, would also be valuable for databases such as the Simons Variation in Individuals Project (Simons VIP), funded by the Simons Foundation.

The Simons VIP aims to study in detail one genetic defect present in autism, to understand the symptoms associated with it and how these develop over time. Its current focus is on deletion or duplication of a stretch of chromosome 16, called 16p11.2.

Abnormalities in this region increase the risk of autism and other neurodevelopmental disorders. About 21 percent of the Simons VIP participants meet strict criteria for autism and the remaining have a range of neuropsychiatric impairments.

The Simons VIP recruited participants through an online community called Simons VIP Connect, which launched in May 2010. At that time, there were 21 known families in the U.S. with the 16p11.2 deletion and 6 with the duplication. Today there are 806 families registered with the project, and 432 of those are residents of the U.S. or Canada who have a confirmed duplication or deletion and have submitted personal information online.

The participants undergo a two-day battery of psychological tests, including intelligence tests and autism assessments, at one of three sites. Researchers have also collected blood samples from 200 individuals with a duplication or deletion, and their families.

Because the participants are scattered across the country and even worldwide, arranging in-person assessments can be challenging and expensive. It can also be difficult for both parents to get time off from work.

More than 200 families registered with the Simons VIP do not actively participate in research because they find travel challenging, says W. Andrew Faucett, co-principal investigator for Simons VIP recruitment and director of policy and education at Geisinger Health System, a physician-led health services organization, in Danville, Pennsylvania. Online assessments would allow more families to participate in the research, he says.

The researchers plan to expand the project and recruit hundreds more participants, notes study principal investigator Wendy Chung, associate professor of pediatrics and clinical genetics at Columbia University in New York. Functional and structural magnetic resonance imaging (MRI) scans must be performed at the clinic, but other information can be gleaned from a distance, and not all tests are crucial.

Online assessments may also make collecting data from the family of the affected individual simpler. Family data can help researchers measure the effect of a particular mutation on a heritable trait — such as intelligence quotient (IQ).

Measuring IQ performance alone isn’t sufficient, because IQ is highly heritable and individuals may be born to parents with high or low IQs, says David Ledbetter, executive vice president of Geisinger Health System.

Comparing the performance of the child who has a genetic alteration with the mean performance of his parents allows researchers to measure the deleterious impact of a mutation even if traits are in the typical range, says Ledbetter. For example, individuals with the 16p11.2 deletion tend to have an average IQ that is two standard deviations lower than the parental mean.

Parents of children with debilitating disease are particularly motivated to participate in research, says Megan O’Boyle, director of the Phelan-McDermid Syndrome Foundation and a parent of a child with the syndrome. However, too many questions can be a burden.

The Phelan-McDermid Syndrome International Registry began with two 100-question online surveys, but O’Boyle and her colleagues soon realized that parents had a difficult time completing them. The foundation is looking for ways to shorten the surveys.

Researchers should also find ways to streamline assessments across studies and registries, so that parents don’t have to answer the same questions over and over, she says.

Data from Patient Crossroads, an online site that hosts Simons VIP Connect and other online disease communities, show that registries with the longest surveys have the lowest completion rates, says Vanessa Rangel Miller, genetic counselor at Patient Crossroads.

Participants also want the ability to save their data and return later, says Rangel-Miller. When participants need to update data frequently — for example, when tracking their child’s height and weight — reminders are particularly effective.

Surveys should also be tailored to the user’s needs. For example, parents shouldn’t have to answer questions about seizures if their child doesn’t have them. Conversely, if a child has sleep apnea, the parents may need to answer additional questions.

The attendees discussed ways to recruit parents into registries, such as providing them with an iPad to enable participation and as a reward. Parents mostly want to know they are contributing and to see the results that pertain to their child, says O’Boyle.

However, sharing data with parents can be challenging if the information was collected for research and not for clinical purposes. Involving clinicians in studies may be difficult and expensive, but it would allow researchers to communicate clinically useful information to parents, says Ledbetter.

Despite the advantages of online assessments, there is the concern that data reported by individuals or their caretakers is less reliable than data collected in the clinic.

To address this, Stanley Nelson, professor of genetics at the University of California, Los Angeles, and his colleagues recruited 107 children from IAN and brought them to the clinic for expert characterization. Of these, 105 had autism using a clinician’s best estimate, 104 had autism according to the Autism Diagnostic Interview-Revised and 97 were diagnosed with autism using the Autism Diagnostic Observation Schedule combined with expert clinical observation.

Another concern is that unsupervised online tests may yield different results than traditional tests. Randy Buckner, professor of psychology at Harvard University, and his colleagues have conducted online assessments for more than 2,000 college students undergoing functional MRI scans as part of the Brain Genomics Superstruct Project.

He and his colleagues observed a small difference in the nonverbal IQ scores between the online version completed at home and the paper-and-pencil test, which is administered in the lab. Participants tend to give up more quickly when faced with difficult questions at home where nobody is monitoring their performance.

To try to replicate the same level of compliance at home, the researchers told participants that they would call them a week after the test to discuss their results.

David Evans, professor of psychology at Bucknell University in Lewisburg, Pennsylvania, found excellent agreement between paper-and-pencil versions and online electronic versions of the Social Responsiveness Scale and the Shipley Institute of Living Scale, which measures intellectual ability.

However, the nonverbal portion of the Shipley IQ test allows paper-test takers to rotate images, which may improve performance. This is not possible for online users. As a result, the online scores for nonverbal IQ were slightly lower than those for the paper versions. One way around this would be to administer the test on iPads, Evans says. He adds that social gaming approaches to online testing might help keep kids engaged in assessment tasks.

Another concern is that individuals who participate in online surveys may not be representative of the wider population. However, individuals who participate in Duchenne Connect — an online community for individuals with Duchenne Muscular Dystrophy — are similar to those who participate in traditional published studies, says Faucett.

The researchers matched the groups based on the nature of their mutation, their age at diagnosis and the age at first wheelchair use.

Another complication of online testing is that many existing assessment tools are copyrighted. Even when publishers do grant permission to administer the tests online, the cost can be prohibitive. Christopher Forrest, professor of pediatrics at the Children’s Hospital of Philadelphia, discussed a suite of free, online-ready instruments available through the Patient Reported Outcomes Measurement Information System (PROMIS), which is funded by the National Institutes of Health.

The instruments are designed to measure feelings, such as fatigue or pain; social and cognitive functions; and perceptions, such as life satisfaction. One goal of using such instruments would be to develop common metrics for universal concepts, rather than developing disease-specific measures.

To use the same questionnaire across a range of abilities, PROMIS relies on computer adaptive testing. Each participant starts with the same question, but his or her response dictates the next question.

For example, an assessment of children’s physical function might begin with a question about sports. A child who can play sports would receive a different second question than one who cannot walk. Each additional question helps the instrument zero in on the correct score.

Computer adaptive testing can also be a valuable tool for assessing IQ. Traditional IQ tests rely on a set number of questions. But fixed-length tests aren’t efficient, says Robert Schultz, director of the Center for Autism Research at the Children’s Hospital of Philadelphia. Bright individuals get stuck answering simple questions, and less skilled test takers may become frustrated by too many difficult questions.

Schultz and his colleagues developed an adaptive test based on the nonverbal matrix reasoning approach. Their measure, the Matrix Adaptive Test, contains 300 questions, but each test taker only sees the subset that best matches his or her ability.

After validating the test against a paper-and-pencil version, the team plans to assess the 2,700 families in the Simons Simplex Collection, a database of families that have only one child with autism, funded by the Simons Foundation. To be able to properly assess IQ, researchers should compare an individual’s IQ with that of his or her parents. However, these data were not originally collected. Schultz’s team aims to assess parental IQs in order to better specify the genetic relationships between IQ and autism risk.

Children who have language difficulties or can’t self-regulate may not be able to complete surveys or assessments, which makes it challenging for researchers to gather data on these individuals. Wearable devices, which can measure autonomic functions, provide a possible solution, says Matthew Goodwin, assistant professor of health sciences at Northeastern University in Boston.

Wockets are open-source wireless devices that contain accelerometers to measure movement. Worn on the wrists and torso, wockets can capture repetitive behaviors often seen in individuals with autism, such as rocking and hand flapping. Researchers can then train an algorithm to recognize those movements.

Q sensors measure skin conductance, which indicates sympathetic response and arousal. These measures can also be combined, to show how rising stress may trigger repetitive behavior or other behavioral outbursts, says Goodwin.

As the day wrapped up, participants identified several groups and companies that might be able to help move the discussion forward.

Matthew Belmonte, assistant professor of human development at Cornell University in Ithaca, New York, has designed video games to help autism researchers collect cognitive data. 23andMe, an online commercial genetic testing company, may also be able to offer perspective and input. The company collects phenotype data through short online questionnaires, in some cases asking hundreds or even thousands of questions per individual.

Researchers may also be able to collect data via routine developmental assessments.

The Children’s Hospital of Philadelphia in partnership with Geisinger Health System has launched an initiative to give parents who are scheduled for well-baby visits an online development questionnaire when they log into their medical records prior to the clinic visit. That model could be expanded to include more detailed developmental and behavioral assessments, Ledbetter says.